[Checkins] [zopefoundation/ZODB] 1a40a7: Use zodbpickle under pypy and 2.7 for a working no...

Wed May 20 18:43:34 CEST 2015

  Branch: refs/heads/master
  Home:   https://github.com/zopefoundation/ZODB
  Commit: 1a40a7e13ac8d72fc59913dd2f381d27ebe863b7
      https://github.com/zopefoundation/ZODB/commit/1a40a7e13ac8d72fc59913dd2f381d27ebe863b7
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-07 (Tue, 07 Apr 2015)

  Changed paths:
    M setup.py
    M src/ZODB/_compat.py
    M tox.ini

  Log Message:
  -----------
  Use zodbpickle under pypy and 2.7 for a working noload. Also add pypy to tox so I can start testing it.

  Commit: f865508ea3f926e01a7b256337290d7642d23d73
      https://github.com/zopefoundation/ZODB/commit/f865508ea3f926e01a7b256337290d7642d23d73
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-07 (Tue, 07 Apr 2015)

  Changed paths:
    M .gitignore
    M src/ZODB/Connection.py
    M src/ZODB/DB.py
    M src/ZODB/broken.py
    M src/ZODB/serialize.py
    M src/ZODB/tests/testBroken.py
    M src/ZODB/tests/testCache.py
    M src/ZODB/tests/util.py

  Log Message:
  -----------
  Checkpoint on ZODB under PyPy: 25 failures, 25 errors (needs my persistent fork). Many of them are printing differences, some have to do with the Python persistent implementation, especially PickleCache

  Commit: 255e256e787b5500783be2ba69f45167c096f7c8
      https://github.com/zopefoundation/ZODB/commit/255e256e787b5500783be2ba69f45167c096f7c8
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-08 (Wed, 08 Apr 2015)

  Changed paths:
    M .travis.yml
    M CHANGES.rst
    M setup.py
    M src/ZODB/ConflictResolution.py
    M src/ZODB/Connection.py
    M src/ZODB/DemoStorage.test
    M src/ZODB/ExportImport.py
    M src/ZODB/_compat.py
    M src/ZODB/blob.py
    M src/ZODB/serialize.py
    M src/ZODB/tests/PackableStorage.py
    M src/ZODB/tests/StorageTestBase.py
    M src/ZODB/tests/testCache.py
    M src/ZODB/tests/testConnection.py
    M src/ZODB/tests/testConnectionSavepoint.txt
    M src/ZODB/tests/testDB.py
    M src/ZODB/tests/testSerialize.py
    M src/ZODB/tests/test_cache.py
    M src/ZODB/tests/test_fsdump.py
    M src/ZODB/tests/testfsoids.py

  Log Message:
  -----------
  All the unit tests pass under PyPy. (The functional tests still have some resource issues.)

The changes were mostly in the Persistent module. Here, the changes were minimal:

- Introduce zodbpickle for PyPy and Python 2.7 to fix noload.

- Centralize the construction of Picklers/Unpicklers to account for the differences between Python2/3/zodbpickle.

- A few extra gc.collect() calls.

- Some minor printing differences in the doctests due to the Python implementation of BTrees.

  Commit: b8fbeb93d49a289454d491ae3704a9f32823bdab
      https://github.com/zopefoundation/ZODB/commit/b8fbeb93d49a289454d491ae3704a9f32823bdab
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-08 (Wed, 08 Apr 2015)

  Changed paths:
    M src/ZODB/blob.py
    M src/ZODB/tests/blob_transaction.txt
    M src/ZODB/tests/testblob.py

  Log Message:
  -----------
  Fix the cleanup of uncommitted blob data under PyPy.

  Commit: cbee2907b42c9409eb170350ccdecf019a4fce40
      https://github.com/zopefoundation/ZODB/commit/cbee2907b42c9409eb170350ccdecf019a4fce40
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-08 (Wed, 08 Apr 2015)

  Changed paths:
    M src/ZODB/blob.py

  Log Message:
  -----------
  Fix the functional blob tests under PyPy/pure-python by avoiding spuriously registering Blob objects as modified each time they are loaded from cache. This is something of a difference in behaviour between the C and Python implementations, but it's also arguably a bug in the Blob class.

  Commit: 8215fdc85c095e372357cde14c854c1c2a986b87
      https://github.com/zopefoundation/ZODB/commit/8215fdc85c095e372357cde14c854c1c2a986b87
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-08 (Wed, 08 Apr 2015)

  Changed paths:
    M src/ZODB/scripts/analyze.py

  Log Message:
  -----------
  One more place to use the PersistentUnpickler. Fixes AnalyzeDotPyTest under PyPy.

  Commit: f1023af2ad6945d3bf251aaa1a3b4faa296013a2
      https://github.com/zopefoundation/ZODB/commit/f1023af2ad6945d3bf251aaa1a3b4faa296013a2
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-09 (Thu, 09 Apr 2015)

  Changed paths:
    M src/ZODB/broken.py

  Log Message:
  -----------
  The Python persistent implementation has caught up enough with the C version that we don't need to list extra special names that shouldn't re-wake Broken objects.

  Commit: 312a493d6722991c4cb4692ed98a7655e166cddc
      https://github.com/zopefoundation/ZODB/commit/312a493d6722991c4cb4692ed98a7655e166cddc
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-10 (Fri, 10 Apr 2015)

  Changed paths:
    M src/ZODB/tests/util.py

  Log Message:
  -----------
  Remove unused helpers, no longer needed.

  Commit: de7f84014d9ead61c1f046bd98f358eae9e38ae0
      https://github.com/zopefoundation/ZODB/commit/de7f84014d9ead61c1f046bd98f358eae9e38ae0
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-10 (Fri, 10 Apr 2015)

  Changed paths:
    M src/ZODB/DB.py

  Log Message:
  -----------
  Better return value for rc in cacheExtremeDetail under PyPy

  Commit: f23065146a5e87162e949ab41dc7e6545312182e
      https://github.com/zopefoundation/ZODB/commit/f23065146a5e87162e949ab41dc7e6545312182e
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-10 (Fri, 10 Apr 2015)

  Changed paths:
    M CHANGES.rst

  Log Message:
  -----------
  Better changeloge entry.

  Commit: b25eb532b0de733e3654815cc287824d0d079490
      https://github.com/zopefoundation/ZODB/commit/b25eb532b0de733e3654815cc287824d0d079490
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-10 (Fri, 10 Apr 2015)

  Changed paths:
    M src/ZODB/DB.py
    M src/ZODB/POSException.py

  Log Message:
  -----------
  Fix unpickling POSError subclasses under PyPy. This fixes a bunch of '<Unprintable POSKeyError>' tests in ZEO.

  Commit: c35f2528a24dc23883a3d8f506628d3c6ae76aae
      https://github.com/zopefoundation/ZODB/commit/c35f2528a24dc23883a3d8f506628d3c6ae76aae
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-11 (Sat, 11 Apr 2015)

  Changed paths:
    M .travis.yml
    M src/ZODB/blob.py
    M src/ZODB/tests/testSerialize.py
    M src/ZODB/tests/test_cache.py
    M tox.ini

  Log Message:
  -----------
  Rollback some changes now that Python persistent handles registering objects during __setstate__ the same way as C persistent.

  Commit: cfec72a18c7b579f63d143c3e3c7a94d48f3e099
      https://github.com/zopefoundation/ZODB/commit/cfec72a18c7b579f63d143c3e3c7a94d48f3e099
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-13 (Mon, 13 Apr 2015)

  Changed paths:
    M CHANGES.rst
    M setup.py
    M src/ZODB/DemoStorage.test
    M src/ZODB/FileStorage/iterator.test
    M src/ZODB/_compat.py
    M src/ZODB/tests/dbopen.txt
    M src/ZODB/tests/testConnectionSavepoint.py
    M src/ZODB/tests/testDB.py
    M src/ZODB/tests/testRecover.py
    M src/ZODB/tests/testSerialize.py
    M src/ZODB/tests/util.py
    M src/ZODB/utils.txt
    M tox.ini

  Log Message:
  -----------
  Add support for Jython. This required only relatively minor test changes.

  Commit: 55167340d847ba53263c4bf5fa03aabc911ade2b
      https://github.com/zopefoundation/ZODB/commit/55167340d847ba53263c4bf5fa03aabc911ade2b
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-14 (Tue, 14 Apr 2015)

  Changed paths:
    M src/ZODB/tests/testConnection.py
    M src/ZODB/tests/testConnectionSavepoint.py
    M src/ZODB/tests/testRecover.py
    M src/ZODB/tests/testSerialize.py

  Log Message:
  -----------
  typos

  Commit: 2807a969147333824ad1af699df3c485d0a5c063
      https://github.com/zopefoundation/ZODB/commit/2807a969147333824ad1af699df3c485d0a5c063
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-04-14 (Tue, 14 Apr 2015)

  Changed paths:
    M src/ZODB/blob.py

  Log Message:
  -----------
  Point to the PyPy issue.

  Commit: f9d6bfd134422d9ca072067fb01faef9aac7777b
      https://github.com/zopefoundation/ZODB/commit/f9d6bfd134422d9ca072067fb01faef9aac7777b
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-05-01 (Fri, 01 May 2015)

  Changed paths:
    M src/ZODB/tests/testSerialize.py

  Log Message:
  -----------
  Add explicit test for noload() of list-based references.

  Commit: 9e0eaaab3469c7bc6936472d0ed5999369c90b9d
      https://github.com/zopefoundation/ZODB/commit/9e0eaaab3469c7bc6936472d0ed5999369c90b9d
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-05-19 (Tue, 19 May 2015)

  Changed paths:
    M setup.py

  Log Message:
  -----------
  Update version pins required for pypy support.

  Commit: 103456c6265290d5d4d54a442339e9b6fd3ca4d5
      https://github.com/zopefoundation/ZODB/commit/103456c6265290d5d4d54a442339e9b6fd3ca4d5
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-05-19 (Tue, 19 May 2015)

  Changed paths:
    M CHANGES.rst
    M setup.py

  Log Message:
  -----------
  Drop mention of Jython since its not currently testable.

  Commit: 165f6408e3dfe3d0b8607020ed14b6a3775e1454
      https://github.com/zopefoundation/ZODB/commit/165f6408e3dfe3d0b8607020ed14b6a3775e1454
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-05-20 (Wed, 20 May 2015)

  Changed paths:
    M .travis.yml

  Log Message:
  -----------
  Remove Travis pin of zope.testrunner; we need at least 4.4.7 in order to run on PyPy3.

  Commit: 5d38a086be4bcf20558d8bcec33cd95d065a7082
      https://github.com/zopefoundation/ZODB/commit/5d38a086be4bcf20558d8bcec33cd95d065a7082
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-05-20 (Wed, 20 May 2015)

  Changed paths:
    M src/ZODB/tests/testpersistentclass.py

  Log Message:
  -----------
  Fix test_new_ghost_w_persistent_class under PyPy3, failing due to a bytes/unicode issue. It seems the cPickleCache under Python 3 accepts unicode objects as the oid, but the pure-Python version doesn't.

  Commit: 0012ca0f9f0f3fe264457930e75ac7b47e002639
      https://github.com/zopefoundation/ZODB/commit/0012ca0f9f0f3fe264457930e75ac7b47e002639
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-05-20 (Wed, 20 May 2015)

  Changed paths:
    M src/ZODB/Connection.py

  Log Message:
  -----------
  Go back to assuming that cache.lru_items and cache.cache_data are in sync. Otherwise we have corruption in the cache and, under PyPy, a CFFI-level memory leak; we want tests to fail in that case.

  Commit: 7372b3102e0fc0fc34be1cc18842c0d1ec65ed4b
      https://github.com/zopefoundation/ZODB/commit/7372b3102e0fc0fc34be1cc18842c0d1ec65ed4b
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-05-20 (Wed, 20 May 2015)

  Changed paths:
    M src/ZODB/_compat.py

  Log Message:
  -----------
  Cleanups in _compat.

  Commit: 8840c09ad350f2483dbc53c9f3b6db92d72a5123
      https://github.com/zopefoundation/ZODB/commit/8840c09ad350f2483dbc53c9f3b6db92d72a5123
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-05-20 (Wed, 20 May 2015)

  Changed paths:
    M tox.ini

  Log Message:
  -----------
  Remove jython stanza copied from another project. Tox apparently has learned this trick.

  Commit: 228604ddc1ad4b741dad716061a48e5a8ea48924
      https://github.com/zopefoundation/ZODB/commit/228604ddc1ad4b741dad716061a48e5a8ea48924
  Author: Jason Madden <jamadden at gmail.com>
  Date:   2015-05-20 (Wed, 20 May 2015)

  Changed paths:
    M src/ZODB/Connection.py

  Log Message:
  -----------
  Connection.setstate passes the object's OID as a parameter to the helper Connection._setstate.

This is a performance optimization for PyPy (profiling shows many recursive calls to Persistent.__getattribute__, and recursive calls seem to trouble the PyPy JIT; this eliminates many of them when loading objects):

"Transaction",                 AFTER       BEFORE
"Add 3000 Objects",             20205      18505
"Update 3000 Objects",          24191      18807
"Read 3000 Warm Objects",       28920      26671
"Read 3000 Cold Objects",       28745      27487
"Read 3000 Hot Objects",        29563      27487
"Read 3000 Steamin' Objects", 1033758     916587

  Commit: 115158a3fc121c40709592f81152ec3b1b701243
      https://github.com/zopefoundation/ZODB/commit/115158a3fc121c40709592f81152ec3b1b701243
  Author: Tres Seaver <tseaver at palladion.com>
  Date:   2015-05-20 (Wed, 20 May 2015)

  Changed paths:
    M .gitignore
    M .travis.yml
    M CHANGES.rst
    M setup.py
    M src/ZODB/ConflictResolution.py
    M src/ZODB/Connection.py
    M src/ZODB/DB.py
    M src/ZODB/DemoStorage.test
    M src/ZODB/ExportImport.py
    M src/ZODB/FileStorage/iterator.test
    M src/ZODB/POSException.py
    M src/ZODB/_compat.py
    M src/ZODB/blob.py
    M src/ZODB/broken.py
    M src/ZODB/scripts/analyze.py
    M src/ZODB/serialize.py
    M src/ZODB/tests/PackableStorage.py
    M src/ZODB/tests/StorageTestBase.py
    M src/ZODB/tests/blob_transaction.txt
    M src/ZODB/tests/dbopen.txt
    M src/ZODB/tests/testBroken.py
    M src/ZODB/tests/testCache.py
    M src/ZODB/tests/testConnection.py
    M src/ZODB/tests/testConnectionSavepoint.py
    M src/ZODB/tests/testConnectionSavepoint.txt
    M src/ZODB/tests/testDB.py
    M src/ZODB/tests/testRecover.py
    M src/ZODB/tests/testSerialize.py
    M src/ZODB/tests/test_cache.py
    M src/ZODB/tests/test_fsdump.py
    M src/ZODB/tests/testblob.py
    M src/ZODB/tests/testfsoids.py
    M src/ZODB/tests/testpersistentclass.py
    M src/ZODB/tests/util.py
    M src/ZODB/utils.txt
    M tox.ini

  Log Message:
  -----------
  Merge pull request #34 from NextThought/pypy

Support PyPy.

Compare: https://github.com/zopefoundation/ZODB/compare/4bc02fde1cde...115158a3fc12