[Zope3-Users] Zope 3 Capacities

[David Johnson]

I wish I could say what we're doing, but I know potential competitors are
listening and these lists are public. I can discuss privately if you would
like to know more.  You're close with the NSA stuff, although on a much
smaller and public scale.

Today, alone, we've processed 15M TX in about 2 hours (2080tps), using a
single MySQL server (not TTW), and MySQL just yawns.  In our experience,
only MySQL and Oracle are capable of doing this (and Oracle only in versions
9 and above).  This is also why I ask so many RDBMS questions on this list.

I do think we will need multiple instances.  I would like to use Zope and
perhaps replace Gadfly with MySQL. Has anyone done this in Zope 3? If so,
how?  I would estimate based upon my preliminary tests that Zope with ZODB
can handle about 300/tps on our hardware.

The versioning feature of Zope alone would blow our clients away.



> -----Original Message-----
> From: Jim Fulton [mailto:jim at zope.com]
> Sent: Friday, January 20, 2006 6:23 AM
> To: David Johnson
> Cc: 'Andreas Jung'; zope3-users at zope.org
> Subject: Re: [Zope3-Users] Zope 3 Capacities
> 
> David Johnson wrote:
> > We're looking at 10-100 billion tx per year stored and performed.
> >
> > Partly I'm trying to gauge where the dividing line is between using the
> ZODB
> > and not, and also estimate how many server instances should be running.
> 
> I'm not sure any transactional database will handle that sort of rate with
> a single database.  We did some tests 2 years ago and with commodity
> hardware, we were able to commit around 50 simple transactions per second
> (tps)
> to a file storage over ZEO.  This is about 60 times slower than you need,
> assuming
> 100 billion tx per year or about 3000 tps.  I imagine you could do
> somewhat better
> than that if you got beefier hardware.
> 
> A *quick* google on transaction rates yielded a fairly old article:
> 
>    http://www.wintercorp.com/rwintercolumns/ie_9903.html
> 
> At that time, most of the databases tested on Unix did less that 100 tps.
> Many of them much less.  Of course that was a long time ago.
> Does anyone know of more recent data?
> 
> Of course, if you can segregate your data, you can get higher
> transaction rates by employing multiple database servers, ZODB
> or otherwise.  I've faily confident that this is what you'll
> need to do.
> 
> Jim
> 
> --
> Jim Fulton           mailto:jim at zope.com       Python Powered!
> CTO                  (540) 361-1714            http://www.python.org
> Zope Corporation     http://www.zope.com       http://www.zope.org