[Zope] Alternate storage for ZODB
Fri, 12 Jul 2002 11:10:07 +0100
On Thursday 04 Jul 2002 3:57 pm, Thierry Florac wrote:
> I don't want to execute queries on Zope's database, but what I'd like t=
> be able to do is to use an external storage, which could provide
> advanced functions such as :
It looks like your concerns are the same as what prompted me to start on=20
> - replication (also available with ZEO),
ZEO doesnt do replication today. It is planned, but for far in the future=
> - partitionning,
of what, and why?
> - handling of very high volumes,
As in file size, or throughput?
FileStorage leads the throughput benchmarks, but even the slowest storage=
not sufficiently slow to make a difference (IMO).
All of today's production storages have serious problems when packing=20
databases with many objects, because they all use an in-memory index of=20
reachable objects. For high volume you need a non-versioning storage (or=20
DirectoryStorage, when its ready)
> In fact, I'm a little bit afraid by the ZODB FileStorage, and by it's
> potential ability to handle very big data files correctly... We have a
> lot of big files to publish and index, and our ZODB will probably have =
> size of 10 GB quite rapidly... Any advise ??
Ive never had a filestorage that big. If memory usage is linear (which I =
is a reasonable assumption) then a FileStorage that big will need (roughl=
1G of ram for normal use, and 2-3G during packing. If thats not a problem=