[ZODB-Dev] python types question

[Tim Peters]

[Jürgen Herrmann]
> i have a backend to establish many2many relations between objects. the
> relations are all bi-directional. for the storage i use a
> PersistentMapping, keys are relation names (strings), values are
> PersistentLists that hold the (globaly uniqe) oids of the related objects
> (strings also).
>
> now my question: regarding performance and having conflict errors in mind
> (an object might well be related to several thousand others, such a
> PersistentList object would be that big then) - is it wise to use these
> two classes or is there something "better" to use? (any hints are ok,
> i'll gladly do the rtfm then :)

The good news is that, since PersistentList and PersistentMapping are
probably the worst choices you could have made, any change you make will be
rewarded with massive improvements at once ;-)

A Persistent{List, Mapping} is a single persistent object.  Its entire state
needs to be saved whenever any part changes, its entire state needs to be
fetched when any containee is referenced, and overlapping transactions
making changes of any kind to one necessarily conflict (this follows from
that any change invalidates the entire state).

ZODB's BTree-based data structures are much more scalable.  You can read
about them in the ZODB Programming Guide:

    http://www.zope.org/Wikis/ZODB/FrontPage/guide/index.html

under Related Modules / BTrees Package.  That link takes you to the ZODB
3.4.1 version of the guide, but the same mildly version-specific stuff, in
PDF format, is included in every ZODB release ("mildly" means this guide
could benefit from a lot more attention than it gets -- it's not updated or
rewritten nearly as often as it would be in a happy world ;-))

An OOBTree sounds natural for the mapping part of what you're doing, and an
OOTreeSet sounds natural for the list part.

You can probably find packages already written for ZODB to manage
relationship mappings, although I'm not up to date on those, so someone else
will have to provide pointers to such.