[ZODB-Dev] Re: Finding out whether an object is in the ZEO or ZODB cache.

[Arnar Lundesgaard]

[SNIP]
> Yes, you definitely dont want to be doing url/object 
> traversal during icp processing. It will kill 
> scalability, add significant latency, and reduce 
> ZODB cache performance.
>
> I wrote the ICP support (and that HOWTO) just before 
> rewriting the ZODB caching system. At the time I was 
> hoping that a cunning  solution to this problem would 
> emerge. Unfortunately it didnt, so we are stuck using 
> crude hacks :-(

Yes, we sortof figured as much. :-/

What we are trying to find is the optimal ICP based load
balancing scheme. Currently we are benchmarking to see
how Squid is performing with the default ICP patch first,
with squid.conf as described in your howto.

Particularly we are wondering what is smartest of
the two options below.

 1. Let Squid wait a while (say something like 0.2 s)
    for all parents to return ICP_OP_MISS or an ICP_OP_HIT.

 2. Always return ICP_OP_SECHO which makes Squid use the
    one that replies first.


> My current implementation hack looks like this.....
> 
> I have a module-level registry which contains the URL
> of big objects currently in memory. My big objects add
> their url into this cache at the point where they are
> accessed. Possibly you dont want to store URL fragments
> rather than whole URLs..
> 
> My ICP handler picks apart the URL by hand, using string 
> method or regular expressions, and checks whether the 
> appropriate URL fragments  are in the registry. It 
> return ICP_HIT if they are.
> 
> It is possible to confuse this scheme by giving it 
> carefully crafted funny URLs.... That doesnt matter 
> - Its only an optimisation.

We where wondering if using 'getPhysicalPath' would
avoid this. We haven't given it too much thought,
but at a first glance it looks like it could work very
well for us.


  Arnar Lundesgaard

----------------------------------
phone: (+47) 982 38 036
mailto:arnar.lundesgaard(at)creuna.no
Creuna as 
Bryggegata 3 
NO-0250 Oslo 
phone office: (+47) 23 23 88 00 
fax: (+47) 23 23 88 50 
http://www.creuna.no/