[Zope-dev] Massive scalability

[Michael Bernstein]

Jimmie Houchin wrote:
> 
> I would like to echo Michael's sentiment and comments. I am at the
> beginning stage of Zope development of a website which will have
> millions of objects of a single object type and multiple of such.

are your objects intended to be indexed by ZCatalog as well,
or are you planning some other method of finding your
objects?

> [snip stuff about mounted databases]
> 
> I think there are many who would like to keep their development within
> Zope using objects. Any information on best use or development
> strategies for such setups would be greatly appreciated.

Thanks for adding your voice!

I also think that applications such as we're discussing
should be possible to deploy using nothing but Zope's built
in capabilities. I understand that Zope has certain
limitations in situations that require many writes to the
database, and I accept those limiattions as being a
neccessary trade-off for a storage strategy that uses an
appending file format. I am still trying to determine if
Zope (especially using the two development options I
outlined) has any built in limitations regarding the number
(or number x size) of objects stored.

I really appreciate Zope's features (especially with
ZPatterns) that allow applications to be developed that are
storage-agnostic, and I feel that this is especially useful
for tying into existing legacy systems, but I don't want to
develop a new application, storing new data, that is tied to
a specific storage methodology such as an RDBMS.

People who wish to customize an application to leverage
their existing legacy data should be free to do so, but I've
noticed that Zope products that have some external system as
a pre-requisite (Worldpilot, ZCommerce, etc.) are deployed
far less often than those which do not (Squishdot, Zwiki,
etc.).

So, again: Has anyone run up against any performance or
other limitations regarding large numbers (hundreds of
thousands or more) of objects stored within the ZODB either
in a BTree Folder or a Rack?

In other words, will the system slow down if you add enough
objects?

Thanks,

Michael Bernstein.