[ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

Thu Jan 27 05:40:58 EST 2011

 On Thu, 27 Jan 2011 10:32:09 +0000, Chris Withers 
 <chris at simplistix.co.uk> wrote:
> On 27/01/2011 03:15, Shane Hathaway wrote:
>>> Okay, so I'll do:
>>>
>>> - e2fsck -f
>
> Hmm, how do I e2fsck a mounted filesystem?
>
> The MySQL filesystem could be unmounted (after I shut down MySQL!), 
> so I
> ran e2fsck on it:
>
> e2fsck -f /dev/sdb1
> e2fsck 1.41.3 (12-Oct-2008)
> Pass 1: Checking inodes, blocks, and sizes
> Pass 2: Checking directory structure
> Pass 3: Checking directory connectivity
> Pass 4: Checking reference counts
> Pass 5: Checking group summary information
> /dev/sdb1: 166/16777216 files (10.2% non-contiguous), 
> 8244979/67107513
> blocks
>
>>> - mysqlcheck -c
>
> mysqlcheck -c dev_packed -u root -p
> Enter password:
> dev_packed.new_oid                                 OK
> dev_packed.object_ref                              OK
> dev_packed.object_refs_added                       OK
> dev_packed.object_state                            OK
> dev_packed.pack_object                             OK
>
>>> What logs should I hunt through and what kind of things am I 
>>> looking
>>> for?
>>
>> /var/log/messages and the like.
>>  Look for kernel-level errors such as
>> block I/O errors, "oops", and system panics.  Any such errors have a
>> chance of corrupting files.  We need to rule out errors at the 
>> kernel or
>> below.
>
> None of the above, /var/log/messages is actually pretty empty for the
> last few days. If it helps, both the app server and database server 
> are
> VMWare virtual machines...
>
>> Next, follow the directions in relstorage/tests/README.txt to create 
>> the
>> 4 test databases.  Then run "bin/test -p -m relstorage".  All tests
>> should pass.
>
> Okay, first problem, the tests only connect to localhost, which means 
> I
> can't exactly test as the app server is one machine and the database
> server is another. However, the two machines are identical, so I 
> setup
> the buildout on the database server with the new test section added.
>
> First up, I get the following failures:
>
> IOError: [Errno 2] No such file or directory:
> 
> '/var/buildout-eggs/RelStorage-1.4.0-py2.6.egg/relstorage/tests/blob/blob_connection.txt'
>
> OSError: [Errno 2] No such file or directory:
> 
> '/var/buildout-eggs/RelStorage-1.4.0-py2.6.egg/relstorage/tests/replicas.conf'
>
> My guess is that these files aren't included by setuptools?
>
> So, I checked out the 1.4.0 tag and added it as a develop egg in the
> buildout.
>
> Now I get:
>
> Running .HFMySQLBlobTests tests:
>    Set up .HFMySQLBlobTests in 0.000 seconds.
>    Running:
>
>    Ran 70 tests with 0 failures and 0 errors in 9.903 seconds.
> Running .HPMySQLBlobTests tests:
>    Tear down .HFMySQLBlobTests in 0.000 seconds.
>    Set up .HPMySQLBlobTests in 0.000 seconds.
>    Running:
>
>    Ran 81 tests with 0 failures and 0 errors in 10.511 seconds.
> Running zope.testing.testrunner.layer.UnitTests tests:
>    Tear down .HPMySQLBlobTests in 0.005 seconds.
>    Set up zope.testing.testrunner.layer.UnitTests in 0.000 seconds.
>    Running:
>      78/269 (29.0%)
>
> Error in test check16MObject 
> (relstorage.tests.testmysql.HPMySQLTests)
> Traceback (most recent call last):
>    File "/usr/local/lib/python2.6/unittest.py", line 279, in run
>      testMethod()
>    File "/home/zope/relstorage_co/relstorage/tests/reltestbase.py", 
> line
> 214, in check16MObject
>      self._dostoreNP(oid, data=data)
>    File
> 
> "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/tests/StorageTestBase.py",
>
> line 202, in _dostoreNP
>      return self._dostore(oid, revid, data, 1, user, description)
>    File
> 
> "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/tests/StorageTestBase.py",
>
> line 190, in _dostore
>      r1 = self._storage.store(oid, revid, data, '', t)
>    File "/home/zope/relstorage_co/relstorage/storage.py", line 565, 
> in store
>      cursor, self._batcher, oid_int, prev_tid_int, data)
>    File "/home/zope/relstorage_co/relstorage/adapters/mover.py", line
> 453, in mysql_store_temp
>      command='REPLACE',
>    File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line
> 67, in insert_into
>      self.flush()
>    File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line
> 74, in flush
>      self._do_inserts()
>    File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line
> 110, in _do_inserts
>      self.cursor.execute(stmt, tuple(params))
>    File
> 
> "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py",
>
> line 174, in execute
>      self.errorhandler(self, exc, value)
>    File
> 
> "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/connections.py",
>
> line 36, in defaulterrorhandler
>      raise errorclass, errorvalue
> OperationalError: (1153, "Got a packet bigger than 
> 'max_allowed_packet'
> bytes")
>
>      196/269 (72.9%)
>
> Error in test check16MObject 
> (relstorage.tests.testmysql.HFMySQLTests)
> Traceback (most recent call last):
>    File "/usr/local/lib/python2.6/unittest.py", line 279, in run
>      testMethod()
>    File "/home/zope/relstorage_co/relstorage/tests/reltestbase.py", 
> line
> 214, in check16MObject
>      self._dostoreNP(oid, data=data)
>    File
> 
> "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/tests/StorageTestBase.py",
>
> line 202, in _dostoreNP
>      return self._dostore(oid, revid, data, 1, user, description)
>    File
> 
> "/var/buildout-eggs/ZODB3-3.9.6-py2.6-linux-i686.egg/ZODB/tests/StorageTestBase.py",
>
> line 190, in _dostore
>      r1 = self._storage.store(oid, revid, data, '', t)
>    File "/home/zope/relstorage_co/relstorage/storage.py", line 565, 
> in store
>      cursor, self._batcher, oid_int, prev_tid_int, data)
>    File "/home/zope/relstorage_co/relstorage/adapters/mover.py", line
> 453, in mysql_store_temp
>      command='REPLACE',
>    File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line
> 67, in insert_into
>      self.flush()
>    File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line
> 74, in flush
>      self._do_inserts()
>    File "/home/zope/relstorage_co/relstorage/adapters/batch.py", line
> 110, in _do_inserts
>      self.cursor.execute(stmt, tuple(params))
>    File
> 
> "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/cursors.py",
>
> line 174, in execute
>      self.errorhandler(self, exc, value)
>    File
> 
> "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/connections.py",
>
> line 36, in defaulterrorhandler
>      raise errorclass, errorvalue
> OperationalError: (1153, "Got a packet bigger than 
> 'max_allowed_packet'
> bytes")
>
>
>    Ran 269 tests with 0 failures and 2 errors in 38.398 seconds.
> Tearing down left over layers:
>    Tear down zope.testing.testrunner.layer.UnitTests in 0.000 
> seconds.
> Total: 420 tests, 0 failures, 2 errors in 58.929 seconds.
>
> This is a little weird, as I have max_allowed_packet set to 16M.
> Should these tests fail?
>
> That said, I don't think this has anything to do with the packing bug 
> as
> I didn't see any exceptions or, in fact, any logging or output at all
> from zodbpack, and the only other exceptions seen were the 
> POSKeyErrors...
>
> cheers,
>
> Chris

 i also had to up max_allowed_packet to 32M to make the tests work.

 best regards,
 jürgen
-- 
>> XLhost.de ® - Webspace von supersmall bis eXtra Large <<

 XLhost.de GmbH
 Jürgen Herrmann, Geschäftsführer
 Boelckestrasse 21, 93051 Regensburg, Germany

 Geschäftsführer: Jürgen Herrmann
 Registriert unter: HRB9918
 Umsatzsteuer-Identifikationsnummer: DE245931218

 Fon:  +49 (0)800 XLHOSTDE [0800 95467833]
 Fax:  +49 (0)800 95467830
 Web:  http://www.XLhost.de