[ZODB-Dev] Re: two level cache

[Atmasamarpan Novy]

Dieter Maurer wrote:
> Jim and Tim convinced me that it would not work (at least not without
> lots of code insprection for C extensions handling persistent
> objects (such as BTrees):
> 
>   The problem: while an object may be read only on the application
>   level, it is not read only below this level:
> 
>     E.g. the ZODB will store the objects load state in "_p_changed".
> 
>   This is not multi-thread safe when different threads can do these
>   updates concurrently (as would be possible when read only objects
>   were shared between connections).

I was looking at code in the meanwhile and I was thinking more in the 
direction of caching of pickles instead of sharing objects among 
connections. It means that moving from L2 into L1 would require to 
unpickle the object and it is certainly not for free. But it still seems 
to me much better than to load it from disk.

Inside ZODB.Connection, the call:

p, serial = self._storage.load(oid, self._version)

would be replaced by

p, serial = self._db.loadFromCache(oid, self._version)
if p is None:
     p, serial = self._storage.load(oid, self._version)
     self._db.saveToCache(oid, self._version, p, serial)

Cache access has to be made thread safe, but we do not want to block it 
during load.

Next thing is that we have to make sure an object is thrown from cache 
when it is invalidated.

--
Atmasamarpan Novy