[Hdf-forum] flush write question

[Quincey Koziol]

Hi Xunlei,

On Jan 12, 2010, at 11:22 AM, Xunlei Wu wrote:

> Hi All,
> 
> I have time series data sets to write out as HDF5. I wondering whether below data flow is proper.

	Looks reasonable to me.

> Or is there any potential memory overwrite.

	I don't see any obvious memory overwrites, but if this is really a loop in your code, then you will need to name the dataset something different each time through the loop, otherwise on your second time through the loop the call to H5Dcreate() will fail.

> For instance, I am not sure whether H5Dwrite() performs the in-time (not delayed) I/O so that I can close the space_id and dataset_id right afterwards.

	Yes, it is a synchronous call and you can close the dataspace and dataset IDs immediately afterwards.

> Also will "delete[] val_f" right after H5Dwrite() ruin the data on the disk?

	No, that's fine.

> Is H5Fflush() redundant in this case?

	Probably, unless there's something else you are doing before the call to H5Fclose().

	Quincey

> Thanks a lot.
> 
> //-------------
> 		//initialize file_id, group_id...
> 		for(int t=start_time_step; t<end_time_step; t++){
> 			dim[0] = X;
> 			dim[1] = Y;
> 			float* val_f = new float[dim[0] * dim[1]];
> 			/*assign data to float val_f[dim[0]*dim[1]]*/
> 			for(int i=0; i<dim[0]; i++){
> 			   for(int j=0; j<dim[1]; j++){
> 				val_f[i * dim[0] + j] = SomeFunction(i, j);
> 			}
> 
> 			hid_t space_id = H5Screate_simple(2, dim, NULL);
> 			hid_t dataset_id = H5Dcreate(group_id, "position", H5T_IEEE_F32BE, space_id, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
> 			herr_t status = H5Dwrite(dataset_id, H5T_NATIVE_FLOAT, space_id, H5S_ALL, H5P_DEFAULT, val_f);
> 			if(status < 0){
> 				std::cout << "H5Dwrite(position) failed. Aborting..." << std::endl;
> 				exit(-1);
> 			}
> 			//H5Fflush(file_id, H5F_SCOPE_LOCAL);
> 			H5Sclose(space_id);
> 			H5Dclose(dataset_id);
> 			
> 			delete[] val_f;
> 		}
> 		H5Gclose(group_id);
> 		H5Fclose(file_id);
> 		...
> //-------------
> 
> Best,
> xunlei
> 
> Hi Xunlei,
> 
> On Jan 10, 2010, at 4:43 PM, Xunlei Wu wrote:
> 
>> / Hi All,
> />/ />/ Reporting a weird behavior involving H5Dcreate2(). The only difference
> />/ between 2 below code sections is using "pos" or "position" as the second
> />/ parameter of H5Dcreate2(). I am wondering whether "position" is a
> />/ reserved word/macro in HDF5. When using "position", all the elements
> />/ written in the .h5 file become -4.3160208E8. Please help.
> /
> 	There are no reserved names for objects in HDF5 files.  This sort of behavior typically indicates a memory overwrite somewhere in the application.  I suggest running it with valgrind to see if you can locate the problem.
> 
> 	Quincey
> 
> 
>> / Best,
> />/ xunlei
> />/ />/ //---- below code writes INVALID data ----
> />/ /*allocate space, initialization, and etc.*/
> />/ ...
> />/ dim[0] = 100;
> />/ dim[1] = 3;
> />/ for(int j=0; j<dim[0]; j++){
> />/ val_f[ j] = ps[i][j].x;
> />/ val_f[ dim[0] + j] = ps[i][j].y;
> />/ val_f[2 * dim[0] + j] = ps[i][j].z;
> />/ }
> />/ />/ space_id = H5Screate_simple(2, dim, NULL);
> />/ dataset_id = H5Dcreate(group_frame_id, "position", H5T_IEEE_F32BE,
> />/ space_id, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
> />/ /* Write data to the dataset; */
> />/ status = H5Dwrite(dataset_id, H5T_NATIVE_FLOAT, space_id, H5S_ALL,
> />/ H5P_DEFAULT, val_f);
> />/ />/ H5Sclose(space_id);
> />/ H5Dclose(dataset_id);
> />/ />/ //---- below code writes CORRECT data ----
> />/ /*allocate space, initialization, and etc.*/
> />/ ...
> />/ dim[0] = 100;
> />/ dim[1] = 3;
> />/ for(int j=0; j<dim[0]; j++){
> />/ val_f[ j] = ps[i][j].x;
> />/ val_f[ dim[0] + j] = ps[i][j].y;
> />/ val_f[2 * dim[0] + j] = ps[i][j].z;
> />/ }
> />/ />/ space_id = H5Screate_simple(2, dim, NULL);
> />/ dataset_id = H5Dcreate(group_frame_id, "pos", H5T_IEEE_F32BE, space_id,
> />/ H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
> />/ /* Write data to the dataset; */
> />/ status = H5Dwrite(dataset_id, H5T_NATIVE_FLOAT, space_id, H5S_ALL,
> />/ H5P_DEFAULT, val_f);
> />/ />/ H5Sclose(space_id);
> />/ H5Dclose(dataset_id);
> />/ />/ />/ _______________________________________________
> />/ Hdf-forum is for HDF software users discussion.
> />/ Hdf-forum at hdfgroup.org <http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org>
> />/ http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org/
>