[ZODB-Dev] Very Large Application Use Case

[Nick Pavlica]

On Wednesday 26 February 2003 9:29 am, Joachim Werner wrote:
> Nick Pavlica schrieb:
> > All,
> >   I know there have been many zope/zodb performance questions in the
> > past, however I haven't found any statistical information regarding t=
heir
> > use in very large applications.  I'm developing the requirements for =
a
> > new open-source project, and estimate the following utilization for a
> > single installation:
> >
> > - 8,000,000 potential users
> > - 400,000 concurrent users
> > - 12,000,000 + transactions in 15 a hour period.
> >
> > I would love to have some feedback from those that have, or are using
> > zope in a "Real" 24x7 production facility under these or larger work
> > loads.
> >
> > Thanks for your feedback!
>
> Hi!
>
> The facts you have mentioned are not the ones that I'd need to give a
> statisfactory answer ;-)
>
> I have not tried using Zope in a scenario that large yet. And I am
> pretty sure that it would take a lot of hardware to serve that many
> users. But I can estimate by multiplying the numbers I know of.
>
> Here are the facts I'd need:
>
> - What will be the read/write ratio?

I would estimate a 60 - 40 split.

>
> The point is: If you just have to serve thousands of read requests Zope
> will manage. You can just add as many ZEO clients as you need.
>
> But as soon as you start writing by the thousands you might get into bi=
g
> trouble.
>
> - Will the content be static or dynamic?
>

95% dynamic

> Static content can be handled by adding caching or dumping pages to a
> file-based webserver. Dynamic content eats up RAM and CPU time ...
>
> - Will it be possible to partition the work load?
>
> An example: Most of sourceforge could scale infinitely because you coul=
d
> just add new boxes for new projects. But things like searching through
> all projects on sourceforge will require interaction between the data o=
n
> the different boxes. This might scale well if you handle things
> intelligently, but it might well not.

Partitioning the app may help things, however one of the design goals it =
to=20
have a unified system.  Something like NDS and multiple Netware servers m=
ay=20
work.

>
> - How "real-time" is the application?

It isn't a "real-time" application, but needs to be able to make data=20
available within a maximum of  10 - 20 seconds.

>
> If it is possible to just mirror parts of the database between separate
> boxes from time to time then things are easier than if you have to go
> into the database in real time and need the latest versions of each and
> every object in the database.
>
> A good starting point would be to look at the physical limitations:
>

Not to worried about the physical limits at this point.

> Let's say you have 150 KB of traffic per transaction, then you'd need a=
n
> AVERAGE data throughput of 260 MBit if I haven't done my calculations
> the wrong way. This means that you need a very good connectivity to
> start with (not even 1 GBit Ethernet will do this if you don't have
> several channels on several servers) ...
>
> The good news are that Zope will not have problems with
>
> - large amounts of data
> - large numbers of users (as long as they are NOT concurrent)

   400,000 concurrent users

> - high read traffic (if you can afford the hardware)
>
> Cheers
>
> Joachim

--=20
Nick Pavlica
EchoStar Communications
CAS-Engineering
(307)633-5237