[hdf-forum] Memory Leak while reading large number of arrays

[Andrew Collette]

Hi,

> I have tried using h5py to see if I could get the same "leak", and indeed I
> had obtained the same behavior (around 50mb of extra memory is allocated by
> the end of the file reading). I have no idea why using the python bindings
> (both our in-house bindings and h5py) I obtain this leak, while using the
> attached script I do not. Does anyone have any ideas?

I'm not able to replicate this from h5py using the file structure you
describe.  How are you measuring the memory leak?  In the attached
script (using Francesc's stat function) I see the memory level slowly
go up from 20M to 30M over the first few hundred timesteps, but then
it levels off at 31040 kB and stays there for the rest of the read.
Presumably this is just HDF5's (or Python's) internal memory
management.  Once it plateaus at 30M I can't get it to go any higher,
even by increasing the number of timesteps to 1000.

If you can replicate the leaking behavior from h5py I'd be happy to
investigate further.

Andrew Collette
-------------- next part --------------
A non-text attachment was scrubbed...
Name: readin.py
Type: text/x-python
Size: 2201 bytes
Desc: not available
URL: <http://mail.hdfgroup.org/pipermail/hdf-forum_hdfgroup.org/attachments/20090630/54c76703/attachment.py>
-------------- next part --------------
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe at hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.