Performance with large ZODB's (was Re: [Zope] Announce - Audio Product)

[ethan mindlace fremen]

--On 05/02/01 20:26:37 -0700 Andy McKay chiseled:

>> Several gigs of RAM?  Cool!  Maybe someone from DC can detail what goes
>> on at startup that might take alot of time.
>
> When you start a ZODB up, an index is built of every object inside the
> ZODB so it that can call it again later, effectively it reads through the
> entire DB.
[...]
> Ive had 1 gig db's and found them a pain to work with from this point of
> view.

2 things:  first, if you use ZEO, then this startup time is pushed to the 
storage server. Your ZClients can start up in seconds, depending on how 
large the ZEO cache is.  For example, with the 2.8GB Zope.org data.fs, ZEO 
clients take about 13 seconds to start up with a 300mb cache and 2 seconds 
to start up with no cache. if you are doing lots of product changes, make 
sure that ZEO_CLIENT is not set so that your products reinitialize.

As far as the memory usage goes, Berkeley Storage will help this 
tremendously.

Hope that helps,
--
-mindlace-
zopatista community liason