[Zope] Zope 2.7.3 Memory Leaks

[Tim Peters]

[Chris McDonough]
> ...
> As I understand it Python has separate memory pools for different
> kinds of objects.  The policy for managing memory deallocation is
> different for each pool.
> 
> I'm no expert about it, I'm just deferring to what I read about it, and
> what I read seems to indicate that releasing the last reference to
> an object doesn't always immediately return the memory
> consumed by that object to the OS.  You probably hit upon a
> case where it does with the above test but I wouldn't assume it's
> indicative of all Python memory deallocation behavior.

It's complicated.  Any request for an object exceeding 256 bytes goes
to the platform malloc().  Python returns such memory via the platform
free().   Whether that in turn "returns the memory to the OS" is
entirely in the hands of the OS and the platform malloc().  It may or
may not.  When it may, whether it does depends on internal malloc()
implementation details, often including the precise history of
malloc() and free() requests.  For example:

    p1 = malloc(50000);
    p2 = malloc(50000);
    p3 = malloc(50000);
    q = malloc(999);
    free(p1);
    free(p2);
    free(p3);

When malloc() works by passing out blocks "low address to high
address", and can only "give back" contiguous high-address chunks to
the OS, then even if malloc() "wants to give memory back" to the OS, a
small object (q) still alive at the highest address passed out so far
will prevent it from returning anything (p1, p2, p3) to the OS.

Objects <= 256 bytes may or may not be obtained via platform malloc().
 Often they're obtained via Python's "small object allocator", which
grabs 256KB at a time from the platform malloc() and carves it up
itself.  These 256KB chunks are currently never given back to the
platform free(), so there's no chance they'll be "returned to the OS"
either.

There are indeed no easy answers here.