[ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?

[Jim Fulton]

On Wed, Oct 13, 2010 at 8:28 PM, Chris Withers <chris at simplistix.co.uk> wrote:
> On 12/10/2010 17:56, Jim Fulton wrote:
>>
>> Currently the only configuration "docs" are in the .xml Alan provided a
>> link to.
>>
>> The best size will depend on the app.
>
> Is there a downside to "too big", provided there's plenty of disk space free
> on the app servers?

Yes. On startup, the cache has to be scanned, and large caches can cause
excessive IO on startup.

...

>> In ZODB 3.10, the cache tracing analysis tools actually work and produce
>> meaningful numbers if you start tracing with an empty cache.
>
> Okay, does that mean the stuff that's documented should be ignored for ZODB
> 3.9?

Cache tracing is busted for 3.9, so yeah. :)

The documentation may be wrong for 3.10 too. I don't know.
I'm working (slowly) on new docs. This will be executable and thus more likely
to be accurate.

>
> I went for the following in the end:
>
> ...
>      cache-size 200Mb
>      client     app
>      wait       on
>    </zeoclient>
>    mount-point /
>    cache-size       50000
>    cache-size-bytes 500Mb
>
> Any obvious issues with that?

No.

>
> One problem I did hit, though, was if I restarted an app server, I got:
>
>  File "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/config.py",
> line 210, in open
>    **options)
>  File
> "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZEO/ClientStorage.py",
> line 395, in __init__
>    self._cache = self.ClientCacheClass(cache_path, size=cache_size)
>  File "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZEO/cache.py",
> line 194, in __init__
>    self._lock_file = zc.lockfile.LockFile(path + '.lock')
>  File
> "/var/buildout-eggs/zc.lockfile-1.0.0-py2.6.egg/zc/lockfile/__init__.py",
> line 76, in __init__
>    _lock_file(fp)
>  File
> "/var/buildout-eggs/zc.lockfile-1.0.0-py2.6.egg/zc/lockfile/__init__.py",
> line 59, in _lock_file
>    raise LockError("Couldn't lock %r" % file.name)
> LockError: Couldn't lock '....zec.lock'

ZEO caches can't be shared between processes.  This is catching that.

I think the lock was added in 3.9 to protect you from cache corruption
due to multiple processes accessing the cache at once.

> ...and yet the app server came up fine.

This is rather disturbing.

> What does this error message imply?

That two processes were trying to access the cache.

> The above did not, however, occur if I stopped and then started the app
> server...
>
> Race condition?

Hopefully, you screwed up somehow and started the process while it was
already running. :)

Jim

-- 
Jim Fulton