[ZODB-Dev] [Enhancement Proposal] Memory size limited Cache

[Jim Fulton]

Dieter Maurer wrote:
> Jim Fulton wrote at 2006-10-6 16:18 -0400:
>> Dieter Maurer wrote:
>> ...

...

>>>   We abused Zope a bit and have build a desktop application with it.
>>>
>>>   One of the main critiques of our customers is too high memory
>>>   consumption. Many have computers with 256 MB memory and
>>>   they do not like at all that our application uses roughly a third
>>>   of it...
>>>
>>>   They would even less like it when there were several processes
>>>   with each of them taking about 60 MB (many of our caches
>>>   are on module level and are shared indeed).
>> OK, so, for a desktop application responsiveness is a big
>> issue.  There the benefit of multiple threads is that you can
>> keep the UI responsive while you do background processing
>> in other threads.  The benefit is responsiveness.
> 
> I am comparing multiple threads with the multiple processes szenario
> you have proposed. Then threads do not give more responsiveness.

And I was comparing multiple threads with a single thread. :)

I would only use multiple processes if I also had multiple
processors. In that case, I'd probably have a decent amount
of memory, which is not your use case.

>> So, in an application like this where there *is* a benefit
>> to having multiple threads *and* you have a lot of read-only
>> data, then I agree that there would be a benefit to sharing
>> the data. Of course, then you have to deal with thread-safety
>> issues that you don't normally have in a ZODB application.
> 
> Thread-safety is not an issue for read-only data.

Yes it is.  An object that is read-only from a persistence
point of view isn't read-only at the Python level.  Objects
often have other data.  Take volatile data as an example.
Or consider object activation and deactivation. If a ghost is
shared among multiple threads, then __setstate__ could
be called from separate threads.

And of course, there's the possibility that buggy software
might try to mutate the objects.  How would you prevent it?
You couldn't wait until commit if there was the possibility that
other threads might see the errant changes.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org