[ZODB-Dev] [ZEO] Storage error with big transactions.

[Tim Peters]

[Andreas Jung]
> I have the following script to emulate a long running writing ZEO client
> by writing 100MB to a page template:
>
> import transaction
>
> pt = app.foo
> while 1:
>     data = '*'*100000000
>     T = transaction.begin()
>     pt.pt_edit(data, 'text/html')
>     T.commit()
>     print 'done'
>
> This script fails badly during during the first commit() call. Is this a bug
> or feature? I am using Zope 2.10.2 on MacOSX Intel.

Probably a bug:

...
> suxmac2:~/sandboxes/zeo-test ajung$ bin/zopectl run test.py
> 2007-02-12 18:18:18 CRITICAL txn.-1610559552 A storage error occurred
> during the second phase of the two-phase commit.  Resources may be in an
> inconsistent state.
> Traceback (most recent call last):
>   File "<string>", line 1, in ?
>   File "test.py", line 8, in ?
>     T.commit()
>   File "/opt/zope/2.10.2/lib/python/transaction/_transaction.py", line 395,
> in commit
>     self._commitResources()
>   File "/opt/zope/2.10.2/lib/python/transaction/_transaction.py", line 503,
> in _commitResources
>     rm.tpc_finish(self)
>   File "/opt/zope/2.10.2/lib/python/ZODB/Connection.py", line 696, in
> tpc_finish
>     self._storage.tpc_finish(transaction, callback)
>   File "/opt/zope/2.10.2/lib/python/ZEO/ClientStorage.py", line 955, in
> tpc_finish
>     self._update_cache(tid)
>   File "/opt/zope/2.10.2/lib/python/ZEO/ClientStorage.py", line 980, in
> _update_cache
>     self._cache.invalidate(oid, version, tid)
>   File "/opt/zope/2.10.2/lib/python/ZEO/cache.py", line 375, in invalidate
>     assert o is not None
> AssertionError

Under the covers "someone" is trying to invalidate a current ZEO cache
entry, and the ZEO cache is complaining because it doesn't believe it
/has/ current data ("assert o is not None").  Best guess for why it
doesn't have it is that it silently refused to add current data to
begin with, because the object pickle was bigger than the size of the
disk file allocated for the ZEO cache.  Offhand I don't recall that
this situation is tested, so it's likely to fail in some way.  One way
to check:  configure the ZEO cache file to be reasonably bigger than
the size of `data = '*'*100000000`.  If the problem goes away then,
that's the cause ;-)