[hdf-forum] hdf5 and data indexing

Mag Gam magawake at gmail.com
Thu Jun 18 07:59:21 EDT 2009


Currently we store historical research data at our engineering lab.
The data is formatted this way on the UNIX filesystem:

$YYYY/$MM/$DD/traversal.tsv.gz

Each compressed traversal.tsv is about 3gig, and we have about 500 of
these files for each day.

For example:
2009/01/01/traversal.tsv.gz
2009/01/02/traversal.tsv.gz
2009/01/03/traversal.tsv.gz
2009/01/04/traversal.tsv.gz
2009/01/05/traversal.tsv.gz


I am using HDF5's group to create the same Unix filesystem structure
in my hdf5 file, but will I be able to access data quickly if I place
some sort of index? Instead of reading the entire hdf5 file?

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe at hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.





More information about the Hdf-forum mailing list