[ZODB-Dev] basic operational documents?

[David McCusker]

Jeremy,

> I know you're aware of the Wiki and, thus, probably the things that
> are linked to from it.  Are you looking for something that isn't
> advertised there?

So far I haven't seen a description of the feature I consider most
important and central to the ZODB architecture.  This is a definition
of how OIDS and object pointers are swizzled, and an exact explanation
of when this activity happens in response to which methods being called.

That's just an example.  I'm interested in the runtime which makes ZODB
work in terms of when bytes get moved around, and how the framework of
classes is organized to distribute the responsibility for different
effects in various objects.

Basically I want to understand the whole thing from top to bottom as if
I had written the whole thing myself, but that's a tall order.

I can't afford to have "I don't know" be my answer when someone asks
me why something happened if we are using ZODB for Chandler persistence,
and some activity is observed that might have a relation to the db.

In order for it to be a good long term solution for OSAF use, I think
I need to understand how all of the source code works, including all
the Python and all the C and/or C++ code, so that I know what happens
in each method I see, and how it relates to the whole system.

This level of knowledge is about, say, 50 times more detailed than a
summary someone might write about each class which explains, "The idea
of class Foo is to cause this part of ZODB to work, because problem
ABC must be solved -- and if we didn't problem XYZ would break us."

I'm just looking for statements about problems and issues being solved
and addressed by the code in ZODB.  Most of the material I see is about
explaining why ZODB even works and why you'd use it, but that level of
information seems inherent in basic storage requirements already (once
you know about one or two storage systems).

> I'm not sure whether an operational document means "How do I write
> applications that use ZODB?" or "How does ZODB work under the covers?"
> or something in the middle like "What is the persistence model exposed
> by ZODB?"  Perhaps you can ask more specific questions if you haven't
> found what you are looking for.

By operational I mean large "what happens and when?"  And I mean at a
couple different levels which are separated by the developer API.  Above
the db interface, one can describe what a developer does with the API
and when, with some explanation of why.

Below the db interface, one can also explain what the implementation
does, and when, along with some explanation of why.

Of the two I'm more interested in the beneath-the-covers side because the
part above the API tends to be similar between storage systems.

It's true that code alone is also a spec for what it does, but it's
not a good spec since it can be very hard to figure out what code means
with hints about what the author expected the code to do.

> In general, I'd say ZODB is underdocumented at all of the levels I
> mentioned.

I'm not complaining, I'm just seeing what I can find.  I won't feel
like I'm doing a professional job for OSAF unless I write my own
summary description of what Chandler persistence does, which a developer
can read without reference to any other documentation if they don't
want to look any further than Chandler docs.

I can't just say we use ZODB and you'll have to go read their docs
in order to understand how to develop for Chandler.  I need to do a
better job than just integrating what Chandler does with ZODB.

If you want, I can start keeping a running list of questions
about why ZODB might be doing.  But the list might be distracting if
in many cases ZODB doesn't ever perform an effect and the question ends
up meaningless.

Today on a walk for exercise, I thought up a scheme that could be used
to figure out which object IDs ceased to exist after aborting a db
transaction, assuming the object IDs were minted by the db, presumably
in serial integer sequence.  Related to that, I have a question about
how ZODB allocates IDs where this is stored in a db.

--David McCusker