[hdf-forum] speeding up h5repack

[Brock Palen]

No chunked layout

Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp at umich.edu
(734)936-1985



On Oct 28, 2008, at 9:00 PM, Elena Pourmal wrote:

> Brock,
>
> It is hard to say for sure why performance is bad.
>
> Do you know if original dataset was chunked?
>
> Try
>
> h5dump -p -H
>
> command on your file  and check for CHUNKED_LAYOUT keyword in the  
> output.
>
> Elena
>
> On Oct 28, 2008, at 1:20 PM, Brock Palen wrote:
>
>> Is there any tweaks that can be done to speed up compressing  
>> already created hdf5 files?
>>
>> For example
>>
>> h5repack -v -i rt_3d_71nm_5micron_hdf5_plt_cnt_0010 -o  
>> lt_cnt_0010_zipped -f GZIP=1
>>
>> Takes 129 Minutes
>>
>> While:
>> gzip rt_3d_71nm_5micron_hdf5_plt_cnt_0010
>>
>> Takes 1.5 Minutes
>>
>> hdf5-1.6.7
>>
>> We don't have szip enabled, but would be interested in trying  
>> (academic work so licensing should not be a problem).
>>
>> Just seemed strange that it took so long,  the uncompressed hdf5  
>> file is from FLASH2.5.
>>
>> Any insight would be nice.
>>
>>
>>
>> Brock Palen
>> www.umich.edu/~brockp
>> Center for Advanced Computing
>> brockp at umich.edu
>> (734)936-1985
>>
>>
>>
>>
>> --------------------------------------------------------------------- 
>> -
>> This mailing list is for HDF software users discussion.
>> To subscribe to this list, send a message to hdf-forum- 
>> subscribe at hdfgroup.org.
>> To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.
>>
>>
>
>
> ----------------------------------------------------------------------
> This mailing list is for HDF software users discussion.
> To subscribe to this list, send a message to hdf-forum- 
> subscribe at hdfgroup.org.
> To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.
>
>
>


----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe at hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.