[ZCM] [ZC] 1847/ 9 Comment "unable to configure zeo client cache tracing"

[Collector: Zope Bugs, Features, and Patches ...]

Issue #1847 Update (Comment) "unable to configure zeo client cache tracing"
 Status Accepted, Zope/bug medium
To followup, visit:
  http://www.zope.org/Collectors/Zope/1847

==============================================================
= Comment - Entry #9 by tim_one on Jul 28, 2005 11:29 am

Re memory use:  sorry, not enough info here for me to guess anything; it's possible; haven't heard other reports pointing at it.  If you want to pursue it, please bring it up on the zodb-dev mailing list, or open a different Collector issue (a Collector item should be exclusively concerned with the problem originally mentioned).  It would help if you could quantify more (for example, what are the actual refcounts both ways; do they increase seemingly without bound over time; ...).
________________________________________
= Comment - Entry #8 by tim_one on Jul 28, 2005 11:09 am

Re Zope svn:  the ZEO cache changes were stitched in (Zope doesn't "own" any ZODB code -- ZODB is a pile of svn:externals wrt to Zope SVN) here, both on Tuesday:

Zope/branches/Zope-2_8-branch
http://mail.zope.org/pipermail/zope-checkins/2005-July/029632.html

Zope/trunk
http://mail.zope.org/pipermail/zope-checkins/2005-July/029633.html

________________________________________
= Comment - Entry #7 by alzhimer on Jul 28, 2005 5:13 am

tim, thanks for the info, I'll surely give it a try (doesn't seem that it made into the Zope-svn-tree, though?).

btw, I have the feeling that the zeo-cache is leaking memory, or at least is using quiet a lot. i have to identical servers with several zeo-storages. for testing i changed the setting for the main storage on one of the servers from 250 MB down to 100 MB. no other changes, both get similar traffic. what we se is, that the client with the smaller cache seems to consume much less memory (we have them restartet by AutoLance, which happens almost three times more often on the server with the bigger cache). this seems to coincide with a much higher refcount for ZEO.cache.Entry.
________________________________________
= Assign - Entry #6 by tim_one on Jul 27, 2005 11:06 am

 Status: Pending => Accepted

 Supporters added: tim_one

I spent most of the last week working on tracing, and there was a mountain of work need to make it aware of MVCC (multiversion concurrency control, new in ZODB 3.3).  The trace files appear correct now, stats.py output too, and simul.py output (when simulating a cache of the same size) so far predicts a hit rate within a percent or two of the observed real-life hit rate.

All this should appear in Zope 2.8.1b1 today or tomorrow.  So far it's only been tested on "artificial" apps designed to stress specific parts, so it would be great if you could give it a try on some real data.
________________________________________
= Comment - Entry #5 by alzhimer on Jul 21, 2005 4:32 am

thanks for the inside. i think, simul.py would not be that important; but it really would help to know if the resulting .trace has correct information, and if the info that stats.py returns can be trusted. at least to me the output does seem to make sense.

Cheers, Sascha
________________________________________
= Comment - Entry #4 by tim_one on Jul 20, 2005 2:55 pm

Nobody working on ZEO tried any of the cache trace gimmicks in ZODB 3.3 or later; the ZEO cache implementation changed entirely in ZODB 3.3; and, there's no test suite exercising any part of the cache-trace code.

Under general principle, "if it's not tested, it's broken", and since the implementation changed completely, there's no reason to hope that much of it still works correctly -- that would require magic.  You just filed a bug report saying that the first few parts you bumped into _didn't_ work, so that's consistent with a lack of magic <wink>.  Since ZODB 3.3 was released well over a year ago, and nobody else filed a bug report, I also have to conclude that few people care about this subsystem.

About cache verification time, it would be better to talk about it on zodb-dev, or open a different Collector issue.  I expect that the cache tracing code here wouldn't help you even if it worked, since AFAIK simul.py doesn't have any code capable of simulating the current ZEO cache design (although I'm not sure about that).

If I were you I'd simply try different cache sizes and measure actual results on my actual data and data access patterns.  At best, if everything still worked, calculating theoretical hit rate on an idealized simulated approximation to the actual implementation (which is the best simul.py could do) would give suggestive but unreliable (gross; over-simplified) advice.
________________________________________
= Comment - Entry #3 by alzhimer on Jul 20, 2005 5:51 am

hmm, in this case, is "believing" == "knowing"? :-) at least, if I set trace=True and run stats.py on the resulting .zec.trace file, i see results...so you say i'd better not trust them, right?

seriously, it would be very helpful to have a tool for cache tuning. i frequently see the problem, that cache verification takes a long time (up to 20 minutes on a 250 MB cache size), which would slow down all other zeo clients as well (the zeo server is quite busy during that time, may be because we have a uncommon huge Data.fs of about 7 GB :-( (we already worked hard to _shrink_ it to that size...)). if i only knew if i would be better off with a smaller cache or not...

as a side note, if the need for a cache verification is discovered, couldn't it be more efficient to simple wipe the cache file and start with a blank?

Cheers, Sascha
________________________________________
= Comment - Entry #2 by tim_one on Jul 19, 2005 5:38 pm

Yes, I believe cache tracing is wholly broken in ZODBs 3.3 and after, and at all levels.  I don't have current plans to look into this, so the best hope for relief is an interested volunteer(s).

Should note that the ZEO cache was rewritten entirely (to support MVCC) after the cache-trace stuff was added, and even if the latter still worked it would no longer have any direct relationship to the way the ZEO cache currently works.  Maybe everything related to cache-tracing should simply be thrown away?
________________________________________
= Request - Entry #1 by alzhimer on Jul 19, 2005 7:40 am

The documention about ZEO client cache tracing is outdated/misleading. In

    Zope/doc/ZEO/trace.txt

it is explained to set the environment variable ZEO_CACHE_TRACE; however, there seems to be no code left that would work that way. In fact, in

    Zope/lib/python/ZEO/ClientStorage.py,

where ClientCache of

    Zope/lib/python/ZEO/cache.py

is instantiated, there is no configurable way to enable the tracing. A simple workaround would be

    #self._cache = self.ClientCacheClass(cache_path, size=cache_size)
    self._cache = self.ClientCacheClass(cache_path, size=cache_size, trace=True)


Unfortunately, I'm not familiar enough with the internals to know how to make this configurable via zope.conf.

By the way, not sure if I'm right, but it looks very much as if

    Zope/lib/python/ZEO/simul.py

isn't working as advertised - no matter what cache size I choose to simulate the collected trace, the calculated hitrate is always 0.0 %.


Cheers, Sascha
==============================================================