[ZODB-Dev] Re: ZODB + ZEO performances problem

[Thierry Florac]

Le jeudi 17 juillet 2008 à 09:38 -0400, Tres Seaver a écrit :
> >> I have one client whose setup is similar (Linux, but using a "big iron"
> >> SAN), where the first phase of incremental repozo backup (the part
> >> checking whether incremental is possible) takes longer by a factor of
> >> two than just doing a full backup.  Putting it on local disk both makes
> >> writes go faster and makes incremental backup feasible.
> > 
> > Actually, I've already seen that "big reads/writes" are efficient on the
> > SAN (more than 40 MB/s) but small ones (many 'seek' and 'read' of 8 Ko
> > as made by ZEO) are not efficient at all ; I'll try to see if putting
> > ZEO databases as well as Zope clients caches on local disks (which are
> > also mirrored) can be more efficient (a hourly cron job can then copy
> > ZEO databases on the SAN for data security, I think...).
> 
> Rather than copying the open file, which can potentially lose a
> transaction which occurs during the copy, I would use the 'repozo'
> script to create backup sets onto the remote drive.


  Hi,

Just to end up this discussion, I think I finally managed to solve all
my problems...
So in fact :
 - I moved all ZEO databases and ZEO clients caches on the local file
system ; even mirrored, read/write performances for ZEO/Zope processes
are far better than what they were on the SAN
 - as this site is an internal site, nightly transactions are unlikely
to occur ; so I can use repozo but a simple file copy may often be
sufficient to save data on the SAN
 - adding a QueueCatalog makes full-text indexing defferable, and
modified objects stores quick
 - system hangs seems to be caused by swapping : as soon as the Zope
process starts to swap on this Solaris box, all threads are completely
frozen ! Adding 2 GB of RAM on the server seems to have solved this
problem...

Thanks all for your help and advises,

  Thierry



-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.