[ZODB-Dev] ZEO's stats.py and simul.py

[Chris Withers]

Tim Peters wrote:

> I don't know.  The last time I know of that anyone tried to use these was in
> 2003, when Jeremy & I were studying alternative cache designs.  

Yeah, I saw Jeremy's wiki page about that. It seemed you guys made a lot 
of progress, did anything get taken forward to a release?

>>stats.py told me I had a 98% hit rate, while
>>http://www.zope.org/Wikis/ZODB/HowtoTraceZEOCache/trace.html says the
>>theoretical maximum is 90%.
> 
> I can't read "90% is probably close to the theoretical maximum" that way.
> Sounds like nonsense anyway, if read literally.  IIRC, 90% was actually just
> the largest "theoretically perfect" hit rate we measured on actual trace
> files from various Zope sites.  A hit rate arbitrarily close to 100% is
> certainly possible.

OK, that makes me think that maybe simul.py has just got out of sync, it 
seems to have attracted less attention...

> Try various sizes and judge results against whatever function you're trying
> to optimize.

Urg. simul.py was supposed to provide an alternative *schniff*

> The obvious one is more disk space required.  If you use a persistent ZEO
> cache, then cache verification time at ZEO client connect/reconnect times
> may also increase proportionately.  Other than those, bigger is probably
> better, and the 20MB (? whatever) default size is much smaller than usually
> desirable (it's left over from days when typical disks were much smaller
> than they are now).  Try, e.g., 200MB.  Like the results better?  Iterate.
> 
> Note that while the ZEO cache is disk-based, it does have in-memory index
> structures taking space proportional to the number of objects cached.  I
> suppose that if the cache file were big enough to hold millions of objects,
> the RAM consumed by those indices could become burdensome.  Haven't heard of
> that happening in real life, though.

OK, so what would you recommend for acheiving best "zodb speed" (ie: 
don't care about disk or memory usage, unless they affect speed) on ZEO 
client servers that are dual processor boxes and have one zeo client per 
processor? How about 2 clients per processor?

cheers,

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk