[hdf-forum] Memory Leak while reading large number of arrays

[Francesc Alted]

A Tuesday 30 June 2009 16:12:22 Bruno Oliveira escrigué:
> Hello all,
> Attached is a sample of code that tries to reproduce the behavior I
> described previously (including a SCons script, but anyone can compile it
> using its own tools). Unfortunately, I have not being able to reproduce it
> fully. The reason is that I was using in-house Python bindings for HD5,
> which I didn't mention because I thought it was irrelevant, but apparently
> it is.
>
> I have tried using h5py to see if I could get the same "leak", and indeed I
> had obtained the same behavior (around 50mb of extra memory is allocated by
> the end of the file reading). I have no idea why using the python bindings
> (both our in-house bindings and h5py) I obtain this leak, while using the
> attached script I do not. Does anyone have any ideas?

No idea, but it seems that PyTables is not bitten by this.  Here it is the 
output of my testbed (attached):

$ PYTHONPATH=. python read-many.py --create-file -s /tmp/read-many.h5
Creating file... /tmp/read-many.h5
WallClock time: 62.215638876
Memory usage: ******* File written *******                                      
VmSize:  151944 kB      VmRSS:   34004 kB                                       
VmData:   29776 kB      VmStk:     180 kB                                       
Reading file... /tmp/read-many.h5                                               
WallClock time: 68.3129489422                                                   
Memory usage: ******* Read iteration #0 *******                                 
VmSize:  150976 kB      VmRSS:   34280 kB                                       
VmData:   28808 kB      VmStk:     180 kB                                       
WallClock time: 74.0996758938
[clip]
Memory usage: ******* Read iteration #9 *******
VmSize:  150976 kB      VmRSS:   34280 kB
VmData:   28808 kB      VmStk:     180 kB

So, at least Python does not seem the guilty here (unless my testbed does not 
check your intended use).

-- 
Francesc Alted
-------------- next part --------------
A non-text attachment was scrubbed...
Name: read-many.py
Type: text/x-python
Size: 2429 bytes
Desc: not available
URL: <http://mail.hdfgroup.org/pipermail/hdf-forum_hdfgroup.org/attachments/20090630/8e33d442/attachment.py>
-------------- next part --------------
----------------------------------------------------------------------
This mailing list is for HDF software users discussion.
To subscribe to this list, send a message to hdf-forum-subscribe at hdfgroup.org.
To unsubscribe, send a message to hdf-forum-unsubscribe at hdfgroup.org.