[Zope-ZEO] How do ZODB transactions work?

[Chris McDonough]

> I have yet to see any documentation of ZODB transactions with more
> detail than this:
>
>    get_transaction().commit()
>
> I suspect this doesn't cover the full range of what's possible.  I
> certainly *hope* there's more to it than that!  Specifically, we need to
> know how sub-transactions work: how they are created, what the semantics
> are, etc.

Here's how I understand it (Jim, please correct me if I'm wrong):

The beginning of a transaction is implicit or can be manually indicated via:

get_transaction().begin()

When you explicitly begin a transaction with begin(), any pending
transaction for the connection is implicitly aborted.

Subtransactions are generally managed implicitly as well (ie. you don't
explicitly specify that you want to begin a transaction with subtransaction
support).

get_transaction().commit(1) indicates you wish to commit a subtransaction in
the context of the current transaction.

get_transaction().abort(1) aborts the current subtransaction after you've
committed at least a single subtransaction.  If you've not committed a
subtransaction prior to calling abort() with a true value, it aborts the
whole transaction.

After you've committed any number of subtransactions,
get_transaction().commit() will commit all the subtransactions you've
committed, and it will commit any uncommitted work in the rest of the
transaction as a final subtransaction.  Likewise, get_transaction().abort()
will abort the entire transaction.

Kapil Thangavelu is trying to document this now for the Product Developer's
Guide.  Let's cross our fingers and hope he has time to release something.

> The context is this: we (currently) have a single-threaded web
> application server with a single connection to a single database.  Most
> of the data in that database is our domain objects, which *should* get
> committed only after explicit user action and careful validation to
> ensure consistency.  However, we also put session data in the database,
> because we want it to persist across server restarts and this is a
> convenient way to do so.  Also, most session objects keep a reference to
> domain objects -- eg. if someone has logged in on a session, that
> session keeps a ref to the corresponding User object.  So if we put the
> sessions in the same ZODB as the domain objects, it Just Works.

You may want to consider mounted databases for this, unless you're using a
nonundo storage for everything.  For more info, see CoreSessionTracking in
the dev.zope.org Current Projects section.  Mounted database transactions
are, however, controlled by the sole global transaction manager.  It sounds
like you want to manually set up two connections to different databases, one
for domain objects and one for session objects.

> However, sessions are committed frequently and implicitly -- typically
> on every hit.  This means that we will commit possibly inconsistent
> domain objects with every hit, because everything is going through that
> sole Transaction object.
>
> So I *think* the model we want is one transaction for the main server,
> and one per session.  Session data would be committed via the main
> server's transaction, and domain objects would be committed by the
> session that made the change -- ie. if we're at a point where the user
> has done some edits to something, has confirmed saving those edits to
> the database, and our code has validated the affected objects for
> consistency, then commit *just* the transaction for that session.

You need two connections for this, and you need to manage them manually.  If
you didn't, you'd be redefining the term "transaction".  I think you really
mean connection.  In Zope, there are a pool of connections shared between
all the threads.  Each connection represents a copy of a database.

You may actually need two separate databases.

>
> So:
>   * does this sound like a sensible way of working?

Maybe.  :-) It may be problematic to successfully associate a request with
an existing (uncommitted) transaction as sessions are continued across
requests.  I guess it's common in Java servlets to do this by storing a
database connection inside a session variable, although this notion is
foreign in the Zope world as a request == a transaction.

>   * can we make ZODB work this way, and if so, how?

By managing separate connections, yes.  How you do this, I can't explain in
detail.  :-(  It may be helpful to get a general understanding of the
problem by reading in particular documentation on Java servlets sessioning
and the storage of connection objects inside session namespaces.  Maybe
that'd turn a light on.

>   * is there a problem if database objects -- our persistent sessions,
>     in this case -- have references to Transaction objects?  if
>     so, can we work around this with __{get,set}state__() methods?

Yes, many problems trying to refer to an object that exists via one
connection but not via another.  I'm sure you love to hear that.  :-)
You'll be exploring generally uncharted waters.