[ZODB-Dev] memory usage recommendation

[Tim Peters]

[DJL]
> I have a project that i want to export to nokia 770 that use zodb
> is there any recommendation to minimize the memory usage of zodb package

Don't store any objects in the database, and don't open any storages or
connections ;-)

Of course those are silly extremes, but they do point to things you have
some control over:

-  When a FileStorage is opened, it builds in-memory data structures
   to help speed the process of loading persistent data.  These
   consume memory proportional to the number of unique objects in
   the database (e.g., there's an in-memory map from an object's id
   (oid) to file offset for the most recent revision of that object).
   The more unique objects you create, the more memory this requires.

-  Each Connection you open maintains an in-memory cache, mapping
   an object id to that object.  This can consume an arbitrarily
   large amount of RAM:  there's no limit on how large an object
   may get, so even a 1-object cache could (in theory) consume all
   the RAM you have.  The fewer connections you have open simultaneously,
   the fewer such caches are created.  You can limit the size of
   the cache via DB's cache_size parameter.  This gives a target
   number of objects.  There is no way to limit cache size by byte
   count, and the target number of objects is a soft limit:  _during_
   a transaction, the cache will hold as many objects as are referenced.
   Between transactions, a cache "garbage collection" pass runs
   to throw away the least recently referenced excess over cache_size
   objects.

   So relatively small transactions with relatively small objects
   should help.

-  Relatedly, use scalable data structures:  when possible, use one
   of the BTree-based containers instead of a persistent list or dict.
   "A BTree" is actually a graph of persistent objects under the covers,
   and is specifically designed to consume memory roughly proportional
   to the number of containees a transaction _accesses_ (rather than
   the-- potentially much larger --total number of containees).