[Hdf-forum] Slow conversion to binary using h5dump

[Elena Pourmal]

Mark,

h5dump performance may be affected by many factors (size of the h5dump default read buffer, chunking sizes of the dataset, compression, etc.)
Would it be possible for you to do h5dump -p -H -d .... to print the header information for the dataset you are trying to export? We may have a better idea what may go wrong.

Thank you!

Elena
On Jan 12, 2010, at 12:15 PM, Jonathan Kim wrote:

> Hi,
> 
> It's different from h5copy.
> 
> What the original data format? BE or LE?
> If it's BE, could you try -b BE and see if any performance difference?
> 
> And could you try with smaller size of HDF5 file?  (under 10GB)
> 
> Also could you try on other filesystems?  (non-parallel as well)
> 
> Since it's the performance issue not a specific bug, more testing results
> would be helpful.
> 
> Thanks.
> 
> - Jonathan
> 
> -----Original Message-----
> From: hdf-forum-bounces at hdfgroup.org [mailto:hdf-forum-bounces at hdfgroup.org]
> On Behalf Of Mark Howison
> Sent: Tuesday, January 12, 2010 10:26 AM
> To: HDF forum
> Subject: [Hdf-forum] Slow conversion to binary using h5dump
> 
> Hi, I tried converting a 108GB HDF5 file to binary using the "-b LE"
> flag in h5dump, but it ran at a crawling pace, only about 4MB/s. This
> is in comparison to an h5copy I did on the same machine (our SGI
> Altix) that ran at 600MB/s. The filesystem is GPFS. Any ideas why
> h5dump is having so much trouble? Is there a conversion phase (to LE)
> that is bogging things down? Thanks, Mark
> 
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum at hdfgroup.org
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
> 
> 
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> Hdf-forum at hdfgroup.org
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org