[ZODB-Dev] Re: Connection pool makes no sense

[Юдыцкий Игорь Владисла]

В Пнд, 02/01/2006 в 15:05 -0500, Tim Peters пишет:
> I'm still baffled by how you get into this state to begin with.  Florent
> explained why earlier, and I didn't see a reply to this part:
> 
> [Florent Guillaume]
> > Huh are you sure? That would mean you have thousands of threads. Or 
> > hundreds or ZEO clients. Or hundreds of ZODB mountpoints.
> > 
> > By itself Zope never uses more than one connection per thread, and the 
> > number of thread is usually small.
> 
> You replied that you had "hundreds" rather than "thousands" of connections,
> but it's a still mystery here how you manage to have that many.  For
> example, Zope defaults to using 4 ZServer threads, so do you specify a much
> larger zserver-threads value than the default?

Well, yes.
We have many (50 - 150) clients that work with zope application at ones.
All application logic is inside Firebird Database, so the application
gets all data to show from the ZSQL methods from RDB connetion.
Activity is not something of stable matter. It jumps up when users are
loging into the system and while regular work sometimes it jumps up and
down. Usually activity looks like alot of small sql requests or stored
procedure calls.

At the time we used Zope-2.7.4 we had zope blocks very often and deny of
serving, while high activity period, so to prevnt blocks we had set up
the big pool-size and zserver threads parameters (we've played with
different values but allways pool_size was larger number for about 1.5
times). It worked for a while. So.. we have moved to zope-2.8.5 and
started to watch every day alot of connections that was opened ones and
never used anymore. It caused DB server overeaten memory and system
swaping. This caused slowing down DB server and growing request queue.
So we started to look through the code of zodb to find out what happens.

> 
> FYI, I added code to clear the cache in _ConnectionPool at the time a closed
> Connection is forgotten.  This will be in ZODB 3.6 final, which in turn will
> be in Zopes 2.9 and 3.2 final.
> 

Okay. Tank you. But it's good not always...
You never said anything good or bad of the idle period. The idea of
having this parameter doesn't seem to be useful for you? IMO it's good
to have a little connection buffer right after the site activity gets
down so in case of next activity jump we have ready connections (may be
with cache, to keep good application perfomance). And after we have
stabilized site load, connection pool may shrink to the size that would
solve the incoming requests (depending of database server load and etc.)
What should be the idle period value - let user to decide. And no need
to the pool_size parameter, or may be the list value of the connection
pool to have ready to server connections. That's why subj was about
connection poll that makes no sense in that architecture. Didn't mean to
be rude... Sorry if i sounded offensive.

> I don't have time for more here now, so if others want more it's up to them
> ;-)
We have discussion that long that would be enough to rewrite Connection,
DB, and utils classes/modules totaly... I would be good testing case.

I've implemented idle logic as i could without deep knowledge of zodb.
And next time we upgrade our system to a new version - will have to
rewrite this logic again.

Anyway, thank you.

> 
> _______________________________________________
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
> 
> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
> http://mail.zope.org/mailman/listinfo/zodb-dev