[hdf-forum] Advice on way to write HDF file for optimal reading.

[nfortne2 at hdfgroup.org]

Andrew,

In that case, you can probably ignore the advice on write performance  
and set the chunks to be as wide as you dare (possibly even the width  
of the dataset).  Still, would it be possible to write the data in row  
order?  That would be significantly faster because the rows are  
contiguous on disk.  In that case you would want to write one "row" of  
chunks at a time (or more).

Thanks,
-Neil

Quoting Andrew Cunningham <andrewc at mac.com>:

> HI Neil,
> 	The situation is 'write-once'/'read-often'.	 That is, the HDF file
> will be written in column order in one data-translation operation and
> closed ( no-compression), then later opened ( and kept open) with
> reading of non-contiguous blocks of rows.
>
> Andrew
>
>
> On Oct 29, 2008, at 3:10 PM, nfortne2 at hdfgroup.org wrote:
>
>> The best way to do this depends on your priorities for performance.  
>>    The most important parameters imfluencing read and write   
>> performance  here are the chunk dimensions, and the raw data chunk   
>> cache (rdcc)  configuration.  Are you using any compression on the   
>> dataset?  The  following advice assumes you are not:
>>
>> T
>
> Andrew

----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe at hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.