[ZODB-Dev] Very Large Application Use Case

[Joachim Werner]

Nick Pavlica schrieb:
> All,
>   I know there have been many zope/zodb performance questions in the past, 
> however I haven't found any statistical information regarding their use in 
> very large applications.  I'm developing the requirements for a new 
> open-source project, and estimate the following utilization for a single 
> installation:
> 
> - 8,000,000 potential users
> - 400,000 concurrent users
> - 12,000,000 + transactions in 15 a hour period.  
> 
> I would love to have some feedback from those that have, or are using zope in 
> a "Real" 24x7 production facility under these or larger work loads.  
> 
> Thanks for your feedback!

Hi!

The facts you have mentioned are not the ones that I'd need to give a 
statisfactory answer ;-)

I have not tried using Zope in a scenario that large yet. And I am 
pretty sure that it would take a lot of hardware to serve that many 
users. But I can estimate by multiplying the numbers I know of.

Here are the facts I'd need:

- What will be the read/write ratio?

The point is: If you just have to serve thousands of read requests Zope 
will manage. You can just add as many ZEO clients as you need.

But as soon as you start writing by the thousands you might get into big 
trouble.

- Will the content be static or dynamic?

Static content can be handled by adding caching or dumping pages to a 
file-based webserver. Dynamic content eats up RAM and CPU time ...

- Will it be possible to partition the work load?

An example: Most of sourceforge could scale infinitely because you could 
just add new boxes for new projects. But things like searching through 
all projects on sourceforge will require interaction between the data on 
the different boxes. This might scale well if you handle things 
intelligently, but it might well not.

- How "real-time" is the application?

If it is possible to just mirror parts of the database between separate 
boxes from time to time then things are easier than if you have to go 
into the database in real time and need the latest versions of each and 
every object in the database.

A good starting point would be to look at the physical limitations:

Let's say you have 150 KB of traffic per transaction, then you'd need an 
AVERAGE data throughput of 260 MBit if I haven't done my calculations 
the wrong way. This means that you need a very good connectivity to 
start with (not even 1 GBit Ethernet will do this if you don't have 
several channels on several servers) ...

The good news are that Zope will not have problems with

- large amounts of data
- large numbers of users (as long as they are NOT concurrent)
- high read traffic (if you can afford the hardware)

Cheers

Joachim

-- 

iuveno AG

Joachim Werner

_________________

Wittelsbacherstr. 23b
90475 Nürnberg

joachim.werner@iuveno.de
www.iuveno.de

Tel.: +49 (0) 911/ 9 88 39 84