[ZODB-Dev] ZODB and ORBit-Python problems

[Greg Ward]

On 18 May 2001, Christian Robottom Reis said:
> How fast is it, Greg? The ZODB/ZEO combo, I mean, regardless of my
> problem? I'm running some tests and reading all objects seems pretty slow. 

It's fast enough.  Our database is currently ~30 MB, ~100k objects.  We
generally access it through ZEO with the client and server on the same
machine, communicating over a TCP socket.  There seem to be two ways to
traverse an entire ZODB:
  * start at the root and follow all pointers (list/dict elements,
    instance attributes) -- ie. object-graph traversal
  * start at OID 0 ('\0\0\0\0\0\0\0\0') and fetch every object
    from the database with dictionary-style lookups on the
    Connection

On my development machine (800 MHz Athlon, 256 MB RAM, Linux 2.2.19,
Python 2.1, and whatever version of ZODB/ZEO Andrew keeps us at --
presumably the latest from zodb.sourcefourge.net, but I dunno), it
takes:
  * ~7 minutes to walk the object graph, type-checking every last
    scrap of data that we hit (wow: I just noticed that the type-
    checking process grows to ~80 MB of memory -- good thing I
    splurged for 256 MB in this machine!)
  * ~4.5 minutes to walk the object graph gathering simple statistics
  * ~2.5 minutes to access every OID (and keep a count of the type()
    of each object, ie. a little bit of extra processing but
    not much)

I haven't done any tests to see how long it takes to *just* walk the
database without doing any ancillary processing.  Might be interesting
to compare the two ways of walking.

> I guess it has to do with the questions: does ZODB guaranteee the same
> object reference for persisted object in a single run? And over runs?

That's a good question.  I would assume *not*, since an object could be
evicted from the object cache; the next time it comes back, it's Python
id() could very well be different.  As for preserving id() across runs:
not bloody likely!

But I am not an expert; this is just educated guesswork and
common-sense.

        Greg
-- 
Greg Ward - software developer                gward@mems-exchange.org
MEMS Exchange                            http://www.mems-exchange.org