[ZODB-Dev] Re: Connection pool makes no sense

[Юдыцкий Игорь Владисла]

В Чтв, 29/12/2005 в 12:27 +0100, Florent Guillaume пишет:
> >>If you see many RDB connections, then it's a RDB problem and not a ZODB 
> >>problem. Something not releasing RDB connections quick enough, or 
> >>leaking RDB connections.
> > 
> > 
> > Not agree. Can you answer the question? Does self.all.remove(c) mean
> > that we WANT to destroy connection instance?
> 
> The self.all.remove(c) in _ConnectionPool attempts to destroy the 
> connection. If something else has a reference to it once it's closed, then 
> that's a bug, and it shouldn't. It should only keep a weak reference to it 
> at most.

But it's nonsense! If weakref exists then some other object has ref to
the obj! And weakValueDictionary is cleaned up automatically when the
last strong ref disappears.

Destroying obj with this logic is absurd:
def _reduce_size(self, strictly_less=False):
    target = self.pool_size - bool(strictly_less)
    while len(self.available) > target:
        c = self.available.pop(0)  <== we have ref to the connection
here, before calling remove
        self.all.remove(c) 

def remove(self, obj):
    del self.data[id(obj)]         <== there is no use to delete obj by
deleting weakref... we just deleting weakref from the
weakValueDictionary!

Try this:
1. add this method to Connection class definition

def __del__(self):
    print 'Destruction...'

then do this:
>>> import sys
>>> sys.path.append('/opt/Zope/lib/python')
>>> from ZODB import Connection
>>> c = Connection.Connection()
>>> del(c)
>>> c = Connection.Connection()
>>> del(c._cache)
>>> del(c)
Destruction...
>>>
See?
You can NOT delete object because _cache keeps reference to it... and
connection remains forever!!! It's cache has RDB connection objects and
they are not closed. Connection becomes inaccessible and unobtainable
trough the connection pool.
That's what I wanted to say. It's definitely a BUG.

> 
> > If not then where in ZODB source code i can see connection destruction?
> > Clearing cache and calling _v_database_connection.close() method?
> Sorry I don't know what a _v_database_connection is, it's not in ZODB or 
> transaction code. If it's RDB code I can't help you.
Don't bother... it's RDB DA handle.
> 
> 
> > You've just caught me on "thousands" but gave no comments on deletion of
> > connection instances... but this is the clue to the topic.
> 
> Even hundreds of ZODB connections is absurd.
> Again, with 4 threads you should never get more than 4 Filestorage 
> connections plus 4 TemporaryStorage connections.
Okay... we moved from Zope 2.7.4, that blocked with small number of
threads and pool_size with high site activity, so we had to increase
those numbers. Anyway in the default configuration of 4 threads and
pool_size of 7 we can watch lots of lost connections, and we now know
both that it's a bug... so we have big pool_size to avoid connection
"deletion" (loosing).
> 
> Florent
> 
--== *** ==--
Заместитель директора Департамента Информационных Технологий
Юдыцкий Игорь Владиславович