[ZODB-Dev] Connection simplification

[Tim Peters]

[Tim Peters]
>> There are two Wiki pages proposing to simplify Connection management:
>>
>>     http://zope.org/Wikis/ZODB/SimplifyConnectionManagement
>>
>>     http://zope.org/Wikis/ZODB/NotesOnPossibleNewZODBConnectionPool
>>
>> Since there are a ton of details not mentioned in either, and the Wiki
>> pages have gotten no feedback, I'm going to ask here.
>>
>> The only thing I'm going to bring up in this msg is a number of optional
>> arguments to DB.open():  temporary, force, and waitflag.  They're not
>> documented, not tested, *AFAICT* they're not used, and all *appear* to
>> be ways to hack around the current hard limit on the # of connections
>> per DB. Anyone want to champion one of them?  For example, does anyone
>> use one of them?  I've said "AFAICT they're not used" about other
>> things, and been surprised.

[Syver Enstad, off-list; I'm replying on-list with his permission]
> I use the wait flag to DB.open. I am using twisted as my application
> server, and it can't block, that would freeze the whole server forever.

The proposals do away with hard limits, so never block.  The current
implementation:

    svn+ssh://svn.zope.org/repos/main/ZODB/branches/tim-simpler_connection

logs a warning if the number of simultaneous connections exceeds pool_size,
and logs a critical problem if that number exceeds twice pool_size, but
keeps creating new connections now matter how many you ask for
simultaneously.

So if your goal is "can't block", it meets that goal as-is.

> I didn't know that it ignored the limit on connections, that would be
> bad.

Here I'm not sure what you need.  I can only guess at waitflag's intent in
3.2 by staring at the code.  That looks clear enough:

    By default, waitflag is true, and DB.open() blocks until it can
    return a connection without exceeding pool_size (it waits for
    someone to close a connection; there's no guarantee that will occur).

    If waitflag is false, and pool_size connections are already in
    use, instead of blocking until a connection is closed, DB.open()
    returns None.

What do you need?  A point of the proposals is that the waitflag=True
behavior doesn't work in practice (a hung app is too often the result), and
there will be no way to get that behavior in ZODB 3.4.

It's certainly possible to hack things in 3.4 so that
DB.open(waitflag=False) would continue to return None sometimes.  I think
that's ugly, but I'll do it if I have to (the ZODB APIs are choked with
"convenience gimmicks" that are unclear, undocumented, & untested; an app
that cares about this could keep track of how many connections it has open
itself).