[ZODB-Dev] session problems

[Tim Peters]

[Florent Guillaume]
> Ok I've dug deeper and now understand the problem. The root cause is in
> the multi-databases support.
>
> The problem is that the Zope startup only closes the main connection it
> had on the root database. The first connection to the TemporaryStorage,
> created and opened during Zope startup, is never closed, so still is a
> synchronizer in its original transaction, but is nevertheless reused in
> other transactins, without a proper synchronizer set up.

Is that a ZODB issue, or a Zope or TemporaryStorage issue?

> When a MountedObject needs to be traversed, it tries to find an existing
> connection for the new database by doing:
>      conn = anyjar.get_connection(db_name)
> where anyjar is the "parent" connection. If there's a linked connection
> for that database, it's returned, otherwise if the multi-database
> already has seen the wanted database, it opens a connection from it, then
> adds it to the "linked" connections attribute (conn.connections) and
> shares this attribute between the two connections.
>
> I that fails, because the connection has never been linked to the new
> database (which is the case during startup code), then the MountedObject
> code does:
>      conn = self._getDB().open()
> Here _getDB() correctly returns a newly instanciated database, which has
> been linked to the other ones in the multi-databases setup (shared
> "databases" dictionnary attribute, ultimately coming from
> Zope2.Startup.datatypes.DBTab.databases).

When do we get to the part that mentions code in ZODB ;-)?  I'm sure you've
found a problem of some kind, but DBTab and MountedObject and
TemporaryStorage and Zope startup aren't part of ZODB, and (sorry!) I really
can't follow this explanation (I'm not familiar with that code).

> Then open() returns a new opened connection for that database. *BUT* this
> new connection is not "linked" to the others (using their .connections
> attribute). This code from get_connections is needed:
>              self.connections.update(new_con.connections)
>              new_con.connections = self.connections
> which would be written, in the context of code executing in MountedObject
> (in _getMountedConnection):
>          except KeyError:
>              conn = self._getDB().open()
>              anyjar.connections.update(conn.connections)
>              conn.connections = anyjar.connections
>          return conn
> But of course really this code doesn't belong to MountedObject. This is
> just the simplest way I could find, if others want to test it.
>
>
> The ".connections" sharing is really funky, apparently all the
> connections opened in the context of the same multi-databases support are
> intended to be present in it.

Yes; see ZODB/tests/multidb.txt, which explains the multi-db design.  This
is also documented in IConnection.connections.

> Why is this access not indirected through the multi-databases support
> in DB itself? 

.connections is a mapping from database name to Connection object; DB
doesn't have any mapping from database name to Connection object, so there's
nothing in DB it could be indirected _to_.  DB.databases is a mapping from
database name to DB object.  A multi-database spans multiple DBs as well as
multiple Connections.  DB keeps track of the DBs involved, and Connection
keeps track of the Connections involved.

> Also I don't understand why open()'s "delegate" attribute is not stored
> as a connection attribute, and close() should reuse it instead of
> obeying a "primary" attribute.  Anyway, I guess historical code, etc.

No, according to SVN, Jim added Connection.open()'s `delegate` argument and
Connection.close()'s `primary` argument late in the game (months after
multidb support was first added).  AFAICT, there are no docs or tests (of
non-default values) for them, and IConnection.close() wasn't changed to say
that Connection.close() grew a new argument.  I don't really know why they
were added.  Jim?  This was the checkin comment on the revision that added
both of them, and they haven't been changed since:

    r30816 | jim | 2005-06-16 16:21:27 -0400 (Thu, 16 Jun 2005) | 9 lines

    Greatly simplified the internal dance to create, open, and close
    connections.   Connections now get their database and related data
    when they are constructed.  They no longer throw away this information
    when they are closed.  (The old design that threw away the database
    and related data did so to avoid memory leaks in Python 1.)

    Fixed a multi-database connection-management bug. (The fix
    necessitated the simplification.)

> I'll let specialistst of the multi-databases decide what to do :)

Me too ;-)