[ZODB-Dev] Persistent Mapping Classes

[Tim Peters]

[Ron Wills]
>  I hope this is the right place to ask about this...

Yup!  Although given the nearly detail-free scenarios you gave us, I hope
you weren't hoping for answers too ;-)

> I have a new project using ZODB (migrating from sql). Everything is
> work well except for one PersistentMapping class. Here's a snippet of
> the class:
>
> class OORTable(PersistentMapping):
>     def __init__(self, oordb, base_class, name=None):
>         PersistentMapping.__init__(self)
>
>         self._table_name = name or base_class.__name__
>         self._columns = PersistentMapping()
>         self._record_count = 0

OK.

>  This OORTable is working as expected, but the attributes _table_name,
> _columns, and _record_count are not always getting committed to the
> ZODB,

What specifically does this mean?  That, for example, the attributes on a
committed OORTable instance don't appear to exist at all, or that they don't
have correct values, or ...?  What do you see or experience that gets
expressed as "not always getting committed"?

> but everything else is (all other objects and their attributes).
>  Here's what I have to far:
>
> 1) One test suite creates a new ZODB populates it and runs several tests.
> If I run this suite on the now existing ZODB the attributes described
> above don't exist,

Does this mean that an attempt to access one of those attributes raises
AttributeError?  That's what it _usually_ means when someone says an
attribute "doesn't exist", but somehow I get the impression that's not what
you're seeing.

> but everything else does...
>
> 2) Another test suite that open the ZODB populates it, then closes
> and reopens it and then runs several tests, these attributes exist
> in the ZODB.

Well, without seeing, or knowing anything about, this code, I doubt there's
much anyone can say.

> Both tests above close the ZODB in the same manner
>
>
> The web interface that utilizes the ZODB, has the same bazaar results.
>
> from ClearWater import MLDB # A thin wrapper around ZODB
> mldb = MLDB.MLDB(_cfg)
> MLDB.mldb = mldb
>
> Works fine until we restart the web server then OORTable attributes are
> missing, but if we do:
>
> from ClearWater import MLDB
> MLDB.mldb = MLDB.MLDB(_cfg)
>
> All works as expected.

Ron, we don't know anything about that code -- nothing at all.  How can we
guess?

Whatever that stuff may be, the semantics should be identical _except_ that
in the first case the calling code holds on to a reference to `mldb` for as
long as the calling code is in scope.  If, e.g., you're using __del__
methods somewhere, that could have major effects.  There is no other
semantic difference in the parts of the snippets you showed except for that
possible difference in the lifetime of the object returned by
MLDB.MLDB(_cfg).

> I've been digging into every document I can find to figure out this
> quirk and need to resolved this before putting it into production
> (which is scheduled for next Wednesday).

Instead of reading docs, I suggest trying to whittle this down to a
smallest-possible failing case.  Are you using any C extension modules for
Python that didn't come with Python or ZODB?  I'm not sure what you're
seeing, but it sounds bizarre enough that low-level C coding errors may be
to blame.

> Here's what is installed:
>
> ZODB3-3.4.1-1
> Python 2.3.4
> Mandrake 10.1

2.3.5 is the current stable release in the Python line, and relative to
2.3.4 had at least one critical bugfix related to garbage collection of
weakly referenced objects.  I'd feel better (and so would you ;-)) if you
switched to 2.3.5 just to eliminate that possibility.  2.3.5 was released
near the start of February, BTW -- there's nothing "cutting edge" about it,
if you're worried about that.

> Please, any help would be greatly appreciated, my nuts are in a bit of a
> vise with this deadline.

FWIW, I've never heard a report like this before -- no idea what's going on.
The sooner you can get a small self-contained failing test you can share,
the sooner the problem will be solved (although I expect you'll discover the
cause yourself during the process of whittling it down).