[ZODB-Dev] Re: Connection pool makes no sense

[Tim Peters]

...

[Tim]
>> FYI, I added code to clear the cache in _ConnectionPool at the time a
>> closed Connection is forgotten.  This will be in ZODB 3.6 final, which
>> in turn will be in Zopes 2.9 and 3.2 final.

[Pitcher]
> Okay. Tank you. But it's good not always... You never said anything good
> or bad of the idle period. The idea of having this parameter doesn't seem
> to be useful for you?

I simply haven't thought about it, and won't have time for that in the
foreseeable future either.  ZODB wasn't designed with hundreds of
simultaneous connections in mind, and for that reason it's not surprising if
odd things happen when you push it that way.

> IMO it's good to have a little connection buffer right after the site
> activity gets down so in case of next activity jump we have ready
> connections (may be with cache, to keep good application perfomance). And
> after we have stabilized site load, connection pool may shrink to the
> size that would solve the incoming requests (depending of database server
> load and etc.) What should be the idle period value - let user to decide.
> And no need to the pool_size parameter, or may be the list value of the
> connection pool to have ready to server connections.  That's why subj was
> about connection poll that makes no sense in that architecture.

Connection management wasn't designed with this use case in mind at all.
Zope typically makes very modest demands on # of connections, and non-Zope
ZODB uses typically even less.  In both, it's considered to be "a problem"
in the app if the app _ever_ opens more than pool_size connections
simultaneously (that's why warning and critical log messages are made when
the app does), it's not expected that pool_size will be set to something
much larger than the default 7, and keeping caches around is important.

> Didn't mean to be rude... Sorry if i sounded offensive.

No offense taken!  I don't mean to sound dismissive either -- you may (or
may not) have good ideas here, but I simply don't have time to think about
them either way.  Hence:

>> I don't have time for more here now, so if others want more it's up to
>> them ;-)

> We have discussion that long that would be enough to rewrite Connection,
> DB, and utils classes/modules totaly... I would be good testing case.
>
> I've implemented idle logic as i could without deep knowledge of zodb.
> And next time we upgrade our system to a new version - will have to
> rewrite this logic again.

This is the right list to discuss such changes.  It would probably help a
lot if more people who cared about these use cases volunteered to help.