[ZODB-Dev] ZODB tools for possible contribution

[Jeff Sasmor]

I could use them in a project I'm doing so if you're
asking for opinions, mine is 'yes'.  
Jeff Sasmor
jeff@sasmor.com
www.netkook.com is an "open Zope CMF site"
----- Original Message ----- 
From: "Greg Ward" <gward@mems-exchange.org>
To: <zodb-dev@zope.org>
Sent: Thursday, August 16, 2001 2:10 PM
Subject: [ZODB-Dev] ZODB tools for possible contribution


> Hi all --
> 
> in the course of using and learning about ZODB and ZEO over the past
> year or so, we have written a handful of useful and nifty tools that I
> think could be useful to everyone using ZODB.  Andrew thinks so too,
> which is why you'll find out-of-date versions of some of them in the
> contrib/ directory of his SourceForge "zodb' project.  Perhaps the right
> thing to do is add these to StandaloneZODB... assuming, that is, that
> StandaloneZODB is poised to take over from Andrew's SF project.  Jeremy,
> Andrew, any opinions on that?
> 
> Anyways, here are the tools we have:
> 
> class ObjectDatabase:
>   * wrapper around the ZODB database/storage/connection troika
>   * "owner" of the database root object (which for us is a
>     PersistentMapping of a handful of one-below-the-root-objects,
>     which we refer to as "database roots")
>   * knows how to open a ZODB in various formats (currently only
>     FileStorage and ClientStorage, but adding more storages just
>     requires adding more '_init_foo()' methods)
>   * the storage type and location (eg. filename or (host, port)) is
>     determined by an environment variable, because we have lots of
>     little scripts that access the database, and we don't want to
>     have to write the same command-line parsing code for every
>     one of them
>   * keeps a list of database roots and the classes that implement them,
>     so if you ask for (eg.) the "user_db" database root, it digs up
>     the UserDatabase instance from the dictionary or instantiates
>     UserDatabase, puts the new root in the database, and returns it
>   * to use it, you subclass and define the _database_roots class
>     attribute -- list of (root_name, root_class) pairs
> 
> class PersistentList:
>   * the obvious analog to PersistentMapping -- it's just a list-like
>     object that sets _p_changed when the list is updated
>   * already included in Andrew's zodb project, just needs to be
>     added to the zope.org CVS
> 
> isinstance(), issubclass()
>   * implementations of these built-ins that work with ExtensionClass
>     with Python < 2.1
>   * these will probably be released with Oscar, a type-checking system
>     I wrote last winter, one of these weeks
> 
> pack_zodb script:
>   * open a FileStorage and pack it
>   * somewhere, we have code that packs a ClientStorage synchronously --
>     ie. it idle-loops until the ".old" file appears, at which point
>     the pack is presumably completed.  This could/should be folded
>     into this script.
> 
> zodb_iterate module/script:
>   * provides an interface to iterate over every OID in a ZODB, calling
>     a callback function for each one that has an object behind it
>   * also provides some functions for manipulating OIDs: pack/unpack/
>     format/parse (these probably belong elsewhere, but they live
>     in zodb_iterate for now)
>   * when run as a script, iterates over a database and prints the
>     OID and repr() of every object, in order by OID
> 
> zodb_census script:
>   * uses zodb_iterate to count the number of instances of every class
>     in the database
>   * has been posted to this list before
> 
> zodb_index script/module:
>   * tool to build/query low-level indeces of a ZODB
>   * current builds 3:
>     - OID -> list of OIDs referenced by it
>     - OID -> list of OIDs that reference it
>     - class_name -> list of OIDs of instances of that class
>   * can do simple reference queries ("show me the list of all objects
>     that reference object 0000000000040fa1") or recursive ("show me
>     the list of all objects referenced by 000000000040fa1, and all
>     objects referenced from them, etc. ad infinitum")
>   * eg. to list all objects reachable from the root:
>       zodb_index.py -fR 0
>     (-f = forward refs, -R = recursive, 0 = OID of the root)
>   * can be used as a module -- provides an API to open and query the
>     indeces
>   * indeces are stored in a separate Berkeley DB file (which can get
>     big -- our 45 MB database has a 20 MB index)
> 
> Do any of these tools sound useful to you?  Do you think they should be
> part of StandaloneZODB?  Turning them into something publicly releasable
> would be several days of work; if there's interest, I'd like to get
> these tools out into the wider world.
> 
>         Greg
> -- 
> Greg Ward - software developer                gward@mems-exchange.org
> MEMS Exchange                            http://www.mems-exchange.org
> 
> _______________________________________________
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
> 
> ZODB-Dev mailing list  -  ZODB-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zodb-dev
>