[hdf-forum] HDF5 file 4x larger than ascii?

Werner Benger werner at cct.lsu.edu
Thu Jun 25 20:57:05 EDT 2009


Now this is interesting: I got an HDF5 file which is 4x larger than
its corresponding representation as "h5ls -rvd" or "h5dump". 

The HDF5 file is 6MB, and available here:

http://sciviz.cct.lsu.edu/data/h5path/path1.f5

Its output by "h5ls -rvd" is 1.4MB:

http://sciviz.cct.lsu.edu/data/h5path/path1.h5ls

And "h5dump" on same file brings it to 1.5MB:

http://sciviz.cct.lsu.edu/data/h5path/path1.h5dump


I'm aware that this kind of data layout is inefficient for the
data stored here; it consists of a time series of just three points
at each time step, each of them stored in some subgroups.

However, I did not expect it to be *that* inefficient such that the
ascii dump is 4x smaller than the corresponding binary HDF5 file 
(using HDF5 1.8.2-post13).

It's not really a performance issue here, since the data file is
still small, and the layout is intended for really large data where
this metadata overhead will become neglible. Still I'm wondering if
there would be a "sufficiently easy" way to reduce the file size
significantly? Maybe there is some "pack all metadata together" property
setting or similar?

Cheers,
	Werner

-- 
___________________________________________________________________________
Dr. Werner Benger <werner at cct.lsu.edu>               Visualization Research
Laboratory for Creative Arts and Technology (LCAT)
Center for Computation & Technology at Louisiana State University (CCT/LSU)
239 Johnston Hall, Baton Rouge, Louisiana 70803
Tel.: +1 225 578 4809                        Fax.: +1 225 578-5362

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe at hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.





More information about the Hdf-forum mailing list