[hdf-forum] chunk cache

[Andrew Collette]

Hi,

> However, when reading the data x-vector by x-vector (in chunk order) it takes 160 seconds, although the cache is setup to hold 32 chunks (2621440 bytes). Its hash table is 3203 entries (next prime larger than 100x cache size as advised in the HDF5 documentation). It appears that most time is spent in user time (over 100 seconds); the actual IO time seems to be fine.

I haven't had this particular issue, but I know there's a  serious
performance problem for chunked writing if the rank of your memory
dataspace doesn't match the rank of the dataset.  It results in a
slowdown of 3x-6x in my case, and manifests in much the same way (all
user time).  If you haven't already, you could try reading in with a
dataspace of [1,1,x] instead of just [x].

Andrew Collette

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe at hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.