[Zope-dev] ZCatalog caching with memcached

[Roché Compaan]

On Mon, 2008-10-27 at 13:41 +0100, Jens Vagelpohl wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> 
> On Oct 27, 2008, at 13:32 , Roché Compaan wrote:
> 
> > On Mon, 2008-10-27 at 13:23 +0100, Jens Vagelpohl wrote:
> >> When it comes to integrating anything in Zope itself I'd choose the
> >> latter.
> >
> > Sure, we're not trying to get this into Zope, we're just sharing our
> > experience and exploring the territory so that one can produce a third
> > party package that really help people with the same use case (which I
> > suspect is quite common one).
> 
> Right, it's perfectly valid to create such a third party package. The  
> discussion just highlights a greater issue. Personally, I don't think  
> it's good practice to focus on the expediency of working around a  
> problem as opposed to tackling the problem directly. The Zope world is  
> littered with add-ons that act as band-aids on real or perceived  
> shortcomings in Zope itself.

Improving the performance of indexes is really really hard. In this case
I really don't think caching is a band-aid, it is a good solution. Even
with optimised indexes, you will find that you need caching to get
reasonable performance if you have a catalog with close to a million or
more documents indexed. Given a large enough catalog, I would argue that
caching is equally as necessary as having a large cache for a ZEO
client.

But caches expire and results get invalidated, and therefor we should
continue to optimise indexes. With some help we should be able to
contribute at this level too.

-- 
Roché Compaan
Upfront Systems                   http://www.upfrontsystems.co.za