[ZODB-Dev] [Enhancement Proposal] Memory size limited Cache

[Jim Fulton]

Dieter Maurer wrote:
> Pascal Peregrina wrote at 2006-10-5 15:45 +0200:
>> ....
>> It would be interesting to have a solution for pure Read-Only Zope instances
>> (using a ZEO architecture, but it might work with a local FileStorage) to
>> allow several requests to be served from the same "Connection object", in
>> order to maximize the in-memory cache (one single cache compared to one
>> cache per connection object), and minimize the invalidations processing. So
>> this is something like an alternate simple transaction management for read
>> only transactions and several Zope requests being handled in the same
>> transaction. I hope that makes (some) sense.
> 
> Interesting...
> 
> We would have a use case for this, too.

We've started moving toward just using a single application thread per process
(with many processes).  There isn't really much advantage in running multiple
threads if you have multiple processes.  The idea above doesn't provide
any benefit if you are only using a single connection.  Of course, this
could get a lot more interesting if Python ever loses the GIL.

To make the idea above work, you would need to arrange for the objects used to be thread
safe.  Many objects have internal non-persistent state.  This state
would have to be managed in a thread-aware way.  This is not practical
in general, but might be made to work for some specific important types,
such a BTrees.

Rather than sharing connections, which I don't think would be practical,
one might be able to arrange read-only connections that shared objects
specially designed and identified to be sharable.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org