[ZODB-Dev] ZEO client cache weirdities: Bug or Caution?

[Chris Withers]

Hi there,

Just been to hell and back taking a site from Zope 2.6.1 to Zope 2.7.2

Kinda concerned that whatever is supposed to invalidate .zec's isn't 
doing so properly.

I have a batch content loading and indexing job, this was falling over 
trying to delete an object object and unindex it.

The error was:

   File "C:\Zope\2.7.2\lib\python\Products\ZCatalog\ZCatalog.py", line 
558, in uncatalog_object
     self._catalog.uncatalogObject(uid)
   File "C:\Zope\2.7.2\lib\python\Products\ZCatalog\Catalog.py", line 
412, in uncatalogObject
     del data[rid]
KeyError: -1363356744

Note that UID...

Now, I fired up a python prompt ZEO client on my desktop.
The first thing I noticed was that the Catalog only appeared to have a 
few of the indexes it should have had. On a whim, I deleted all the .zec 
files I could find and restarted... everything fine :-)

...except, the UID for the object is question was now -1363356714.

Bwah?!

After much checking of BTrees (http://zope.org/Collectors/Zope/1450) and 
gnashing of teeth, it finally dawned on me to blow away the .zec's on 
the troublesome client.

Bingo, the delete worked fine.

Now, this has me MORE than a little worried. What if something had got 
committed with that rubbish. (Plenty has, but I haven't seen any ill 
effects...)

Especially as I often connect from the client on my desktop to my test 
storage server, and then flip it onto the live storage server (which was 
why it only showed a few indexes...)

What's the deal here? Should I just use temporary ZEO client caches and 
run out of temp disk space every so often? Is this a bug in ZEO that 
should be fixed? Do I only need to blow away .zec's when I change the 
storage server I'm connecting to? What about versions of Zope?

Waaaaaaaaaaaaaaagh....

Chris

-- 
Simplistix - Content Management, Zope & Python Consulting
            - http://www.simplistix.co.uk