[ZODB-Dev] Extenuated memory

[Dieter Maurer]

Antonio Beamud Montero wrote at 2006-3-23 09:26 +0100:
> ...
>But my server doesn't free any memory, now it uses 55Mb of resident
>size... How I can debug this "references"? 

It has no chance -- and this is neither the fault of Python nor
of the ZODB but of "C"'s standard memory management:

  "C" does not use memory compaction (if it would, development
  would be much more difficult, as any memory use would need
  communication with the compacter (in the form of: I am currently
  using this memory, do not move it aways while I do).

  Thus, uses memory snippets cannot be moved in order
  to have all free memory at the end of the heap.

  "*nix" has 2 ways to allocate memory:

   *  the "brk" system call that can add and remove
      memory from the end of the data/heap segment

   *  and "mmap" which can add large memory blocks to
      the process address space.

  "mmap" is only reasonable for large blocks, smaller blocks
  are usually allocated via the "brk" system call --
  and this memory can only be given back to the OS if
  a contiguous block is free at the end of the data/segment heap.
  The "C" level memory management decides when to give
  memory back this way (and how large the free block must be
  before it does). As occupied memory cannot be moved, its
  chances to give back large chunks are limited...


This problem plages *ALL* C programs that intermittendly
require large amounts of memory (but know in very large blocks
which could use "mmap").

-- 
Dieter