[ZODB-Dev] Opening 7 Connections hangs (and __del__)

[Christian Reis]

(I made a big mess with the CC: line here. Sorry.)

On Fri, Jan 31, 2003 at 09:55:25AM -0500, Jeremy Hylton wrote:
> You're very thorough by you're overlooking the obvious :-).  You did
> all that work to get a C stack trace for the program, but you didn't
> look at the implementation of open()!  So you've missed an obscure

Well, it was hanging so I can't get a regular traceback --- how do you
think I found out the problem *was* in db.open() in the first place
<wink>? It's always neat to debug larger systems - anything can go wrong
anywhere in that thick layer (for instance, the bottom of that stack
trace lives inside PyGTK and the middle in GTK+ - I started off 
blaming JamesH and Johan!)

> A database use a thread lock to prevent more than N threads from
> opening database connections at one time.  The number is configurable
> via the pool_size argument to the DB constructor.
> 
> The reason __del__ isn't getting called is that Connections are never
> destroyed, they're just recycled.  The idea is that the cache for a
> previously used connection may contain objects useful for the next
> client to call open().

It seems that my connections are never reused. This is my use case: I
click on a button and open a new connection (db.open()), sending it into
a GUI dialog. The dialog closes, it is hidden, and my connection is
released (or rather, the wrapper, read on).

I have a wrapper class around connection which is the only instance
that holds a reference to Connection in my application. When this
wrapper is released, it's __del__ *is* called, so I assume the
connection it holds would be released and reused, according to your
explanation.

It never is. If I don't call self._conn.close() (yes, _conn is the
"real" connection instance) in the destructor of the wrapper, it hangs
after 7 cycles of open/close window.

Now I'm not sure if this is a bug, if I should call close() for my use
pattern, or if I should specify "temporary=1" to db.open. The last seems
the simplest, of course. 

Or is there another magic method() to connection that I need to call
that tickles it into being reused?

> We're planning to eliminate at least the locking part of the
> connection pool in ZODB4.  Instead of blocking, you'd just allocate a
> new connection.  The feature exists in its current shape because Zope

This can be done today with db.setPoolSize(), which is another
workaround for the problem. However, I'm still a bit lost as to why *my*
connections aren't being reused.

I'm not against having connection pools in ZODB at all; indeed, many
relational databases have them and it seems to speed up things
significantly. Most of them also allow you call close() explicitly,
which AFAICT returns the connection to the pool, for reuse. So why can't
we call close() here? :-)

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL

Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL