[ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

Marius Gedminas marius at gedmin.as
Thu Dec 10 11:54:57 EST 2009


On Thu, Dec 10, 2009 at 10:08:36AM -0500, Jim Fulton wrote:
> On Thu, Dec 10, 2009 at 5:58 AM, Marius Gedminas <marius at gedmin.as> wrote:
> > On Wed, Dec 09, 2009 at 01:04:03PM -0500, Jim Fulton wrote:
> >> On Wed, Dec 9, 2009 at 12:06 PM, Marius Gedminas <marius at gedmin.as> wrote:
> >> ...
> >> > (Supporting both ZODB 3.8 and 3.9 is kinda tricky, but with some very
> >> > ugly hacks I managed.)
> >>
> >> This sounds like something that needs to be fixed. Can you share some of the
> >> issues you ran into? (Or maybe file bugs reports.)
> >
> > Two issues only:
> >
> >  * In ZODB 3.8 PersistentDict and PersistentMapping are unrelated
> >    classes, so if I want to have adapters for both, I have to register
> >    them separately in ZCML.  In ZODB 3.9, PersistentDict is an alias
> >    for PersistentMapping, so I get a ZCML configuration conflict error
> >    if I have adapter directives for both.
> 
> OK, it's impractical to do anything about this.
> 
> >
> >    Solved by defining a decoy class for one of the adapters so they
> >    don't conflict.
> 
> Hm, I'm not sure I follow this. 

Code:

    if PersistentMapping is PersistentDict:
        # <...snip large comment explaining this...>
        class DecoyPersistentDict(PersistentMapping):
            """Decoy to avoid ZCML errors while supporting both ZODB 3.8 and 3.9."""
        class PersistentDictState(PersistentMappingState):
            """Decoy to avoid ZCML errors while supporting both ZODB 3.8 and 3.9."""
            adapts(DecoyPersistentDict, dict, None)
    else:
        class PersistentDictState(PersistentMappingState):
            """Convenient access to a persistent dict's items."""
            adapts(PersistentDict, dict, None)

ZCML:

    <adapter factory=".state.PersistentDictState" />
    <adapter factory=".state.PersistentMappingState" />

> You could have used a different
> approach of providing the second adapter registration as an override.

Oooh, neat hack.  But I'm not sure I like it better than the one I
implemented.

> >  * In ZODB 3.8, the 'version' argument of ClientStorage.history (as well
> >    as other kinds of storages, I suppose) is mandatory.  In ZODB 3.9
> >    it's gone.

I went back and corrected my statement (originally it referred to
FileStorage.history, and I was thinking about ClientStorage in my
parenthetical statement), and then I of course forgot to remove the
now-obsolete parenthetical statement itself.

> It's mandatory in ClientStorage and optional in FileStorage. (MappingStorage and
> DemoStorage don't have a history method in 3.8.
>
> I suggest we treat this as a bug in 3.8 and make the version argument
> to ClientStorage's history method optional in a 3.8 bug fix release.

+1

> >    Solved by peeking into the method signature with inspect.getargspec()
> >    and supplying a version only if it's needed.
> 
> Ick. :)

:)

> >> In particular, I think it should be a goal that it isn't too hard to
> >> write code that works with both ZODB 3.8 and 3.9.
> >
> > Normal code doesn't usually play with storage.history, I'd say, so I'm
> > not too peeved by having to do various contortions.
> 
> Zope 2 had (has?) a standard view for looking at an object's history
> that I found very useful. I wish there was one for Zope3.

Well, there is IStorage, and I think all storages (except for
FileStorage) implemented it faithfully (oid,
version-that-got-removed-in-3.9, size=1).  FileStorage in 3.8 violated
the interface by renaming size=1 to length=1, but it's fixed in 3.9.

I personally would like to be able to have size=None imply "unlimited,
give me as much as you've got"), but 9999999999999999 works just as well
in practice.

What I miss _a lot_ is a way to look at the last several transactions.
There's the iterator API, but it goes from oldest-to-latest, and when
you've got a large DB, that takes a while (85 seconds for a Data.fs with
922737 transactions and 8 GB of data, with 100 MB/s read throughput).

> History can be very useful in my experience.

Oh yes.  This is why I stopped packing my production Data.fs'es and
first started using storage.history() with a couple of nice wrappers
from debugzope shell, and, later, convinced a coworker to write
ZODBBrowser.

Marius Gedminas
-- 
I used to think I was indecisive, but now I'm not so sure.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20091210/55c3ccaa/attachment.bin 


More information about the ZODB-Dev mailing list