[Zope-ZEO] Re: ZEO and ZODB timing results

[Jim Fulton]

Greg Ward wrote:
> 
> On 02 May 2000, Jim Fulton said:

(snip)

> > >   * for ZEO "query processes", the bulk of the time was the first query
> > >     (fill the cache, presumably)
> >
> > This probably unfairly penalizes ZEO. In a realistic usage scenario,
> > most queries will be fed from the local cache.
> 
> Until the problem size exceeds the cache size, that is.  Since this is a
> pretty small database (so far!), I expect the whole thing fits in the
> cache.  This is demonstrated by looking at more detailed timing for the
> "query process library" script.  It does 7 queries, all of which involve
> walking some portion of the whole object graph.  The latter 3 queries
> are more detailed and require a deeper graph traversal.  Here are
> results for a single run with each of ZODB and ZEO:
> 
>               ZODB                  ZEO
> query:    cpu       elapsed    cpu      elapsed
>   1       0.50      0.52       0.21     14.26
>   2       0.15      0.15       0.15      0.49
>   3       0.15      0.15       0.15      0.54
>   4       0.14      0.14       0.13      0.15
>   5       0.25      0.25       0.26      2.87
>   6       0.17      0.17       0.16      0.17
>   7       0.18      0.18       0.14      1.92
> 
> Observations:
>   * the initial query with ZODB has a bit of a delay as the cache
>     is filled in; both filling the cache (and the subsequent graph
>     traversals) appear to be CPU-bound
>   * the initial cache-fill is *much* more expensive with ZEO, and
>     clearly not CPU-bound.  This is as expected, and not really
>     a big deal: in a realistic situation, one process would live
>     for days or weeks and handle many thousands of queries.
>   * the two queries that involve an extra 2-3 sec delay under ZEO
>     do so consistently -- this is not a random network effect.
>     I suspect it might be because these two queries have to go
>     a bit deeper in the object graph, so there's some more cache-
>     filling going on here.
>   * this whole exercise is rather bogus on two accounts: the whole
>     database (presumably) fits in the client-side cache (which
>     is an unfair advantage to both ZODB and ZEO), and no indexing of
>     any kind was done (an unfair disadvantage, unless you're working
>     for Versant and trying to highlight the fact that Z* don't
>     do any indexing for you)

The philosophy used by ZODB is that features like indexing or security
should be provided at a level above the database.  There *are* indexing
facilities available for and designed to work with ZODB.  At a low level,
there are BTree classes that can be used with ZODB directly. There is
a system in Zope called ZCatalog that could probably be adapted to
non-Zope applications to provide higher-level indexing capabilities.

> I need to re-run this on progressively larger problem sizes, until we
> burst the cache and performance on all queries is abysmal.  Then I need
> to play with indexing schemes.

It occurs to me that there is another caching option that you aren't 
available.  ZEO has an additional disk-based client cache. The size of this
is about 40 megs by default. Also, by default, this cache is not reused 
accross runs, but you can make the cache persistent, which means that
you'll get benefits from run to run.

(snip)

> > Wow. These times are really bad.  I ran a test that created a 100MB
> > database on a crappy old 100Mhz mobile non-MMX pentium-based laptop.
> > My time with the ZEO you have was about 2400 seconds. I'd really like to
> > get a copy of your test script so I try to understand why your numbers
> > are so bad. Obviously, I'd like you to try out a new ZEO as soon as it's
> > available.
> 
> OK, I'll bundle up our code and send it to Jim.  Shhh, don't tell CNRI's
> Crack Legal Team(TM) that I'm doing this.

Your secret is safe with me. :)

Jim

--
Jim Fulton           mailto:jim@digicool.com   Python Powered!        
Technical Director   (888) 344-4332            http://www.python.org  
Digital Creations    http://www.digicool.com   http://www.zope.org    

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission.  Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.