[ZODB-Dev] zeodb limitation with zope 2.12

[Chris Bainbridge]

2009/5/14 Dganit David <Dganit.D at puresight.com>:
> We are planning  a web application that saves the data of up to 1,000,000
> users.
>
> I would like to know the limitation of zodb.
>
> Currently we are using   ClientStorage with zeo,

In my experience building a large scale application, ZEO did not scale
with my application in the way that I had hoped. It worked great with
a few hosts accessing the ZEO server in parallel, but when scaled up
the ZEO server itself quickly became a bottleneck which had to be
designed around. The thing that surprised me was that the server
itself became the bottleneck with only 4 hosts active - it obviously
depends on the access patterns of your application. A single ZEO
server may not serve the needs of 1 million users if you have a large
number accessing the application concurrently, so I suggest that you
think about multiple servers, replication, and dividing the data
itself up to enabling parallel distributed access before starting to
code the application. And ensure that you have a test suite that will
simulate the large scale requirements from the beginning.

The only other limitation was packing - when the database grew to
several GB it became a lengthy process to pack, and eventually packing
became impossible due to the large memory requirements (I posted to
the list at the time, and discovered that this was expected
behaviour).

Once you understand that ZEO is not some magic solution, and that you
still need to think about all the usual database problems, then things
will fall into place quite easily ;-)