[ZODB-Dev] BerkeleyDB and multi-cpu

[sean.upton@uniontrib.com]

Would it matter if the underlying libraries could since the bsddb3 libraries
for python are likely subject to Python's limitations?  I would think that
it wouldn't scale any better on multiple CPUs because the bottleneck is the
same.

Aside from a theoretical future quorum StorageServer arangement to spread
load across several uni-processor boxes, I can't see may other ways to
further scale the ZSS component to spread load across CPUs, other than just
getting a faster Uni-processor box.

On the other hand, Multiple CPUs under most OSs can improve device I/O
performance in almost any application, I can't see why the raw file I/O
speed might increase on a dual CPU box over an equivanlent single CPU box...
and since ZEO is more I/O and storage intensive than CPU intensive, it makes
sense.  There are likely limits to that theory, though: for example, a 2CPU
400MHz Sun E450 server would likely not outperform a much cheaper single-CPU
1GHz+ Intel or AMD based box with equivalent RAM, storage, and network
hardware.

If one was really concerned about scaling a ZSS/ZEO now, I would think that
the appropriate solutions would be:
1 - Use fast hardware: Fast dual-cpu (for I/O improvement only), HW RAID,
lots of RAM
2 - Use a Kernel with a quick raw TCP performance; I assume that Linux 2.4
is a good choice; eventually, the Linux kernel will include David Miller's
zero-copy write patches that are being used with TUX; they likely couldn't
hurt.
3 - Distribute load between 2 ZEO server instances on the same dual-CPU
machine serving 2+ different storages; I know this is a pain, as it requires
support for it be built in at the application level...
4 - Use a dedicated network or VLAN for ZEO alone; we do this to guarantee
the ZEO communication it's own pipe.

Sean

-----Original Message-----
From: Jimmie Houchin [mailto:jhouchin@texoma.net]
Sent: Monday, July 30, 2001 12:20 PM
To: zodb-dev@zope.org
Subject: [ZODB-Dev] BerkeleyDB and multi-cpu


I was just curious if BerkeleyDB utilized multiple processors on
multi-cpu machines?

Or would this be dependent on the Berkeley Storage implementation?

This would be interesting to know if one was using the Berkeley Storage
as the backend for a ZEO setup.

A multi-processorable Berkeley Storage backending ZEO with many ZEO
clients would be most excellent for scaling.

Thanks for any help.

Jimmie Houchin

_______________________________________________
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://lists.zope.org/mailman/listinfo/zodb-dev