[Checkins] [zopefoundation/ZEO] 062cbe: If vote fails with a conflict error, invalidate th...

Thu Jul 7 17:04:23 CEST 2016

  Branch: refs/heads/uvloop
  Home:   https://github.com/zopefoundation/ZEO
  Commit: 062cbecc273b58ebbebd26eec1e8abae89795d0d
      https://github.com/zopefoundation/ZEO/commit/062cbecc273b58ebbebd26eec1e8abae89795d0d
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-01 (Fri, 01 Jul 2016)

  Changed paths:
    M src/ZEO/ClientStorage.py

  Log Message:
  -----------
  If vote fails with a conflict error, invalidate the corresponding cache entry

This used to happen via the server calling serialnos, but ZEO 5
servers no longer do this.  For the reason for invalidating the cache
entry, see the new comment.

  Commit: 5e6fd05f0ae58eb30179a9ec47c83c5a00bd35d6
      https://github.com/zopefoundation/ZEO/commit/5e6fd05f0ae58eb30179a9ec47c83c5a00bd35d6
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-01 (Fri, 01 Jul 2016)

  Changed paths:
    M src/ZEO/cache.py
    M src/ZEO/interfaces.py

  Log Message:
  -----------
  Make the cache thread safe again. :/

There are some cases where we want to touch the cache outside of the
I/O thread. Including:

- ClientStorage wants to to invalidata cache entries if it gets a
  conflict error in voting.

- loadBefore can probably be optimized by checking the cache in the
  client thread. (This will be safe for loadBefore, because the intent
  in practice is never to load current data.

  Commit: 020a9f8c671ff5af253e8924e2f012b9a891164b
      https://github.com/zopefoundation/ZEO/commit/020a9f8c671ff5af253e8924e2f012b9a891164b
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-01 (Fri, 01 Jul 2016)

  Changed paths:
    M src/ZEO/StorageServer.py
    M src/ZEO/asyncio/server.py
    M src/ZEO/asyncio/tests.py
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/protocols.test
    M src/ZEO/tests/testZEO2.py

  Log Message:
  -----------
  Simplified error handling during vote

ZEO's vote error handling was extremely and weirdly complicated.
There's a hysterical explanation: Originally, the ZEO protocol was
synchronous and mirrored the storage API. That was too slow, so store
was made asynchronous.  But then there was no way to return new
serials, so we added a serialnos client API that the server called
during TPC. This was used both to serials and errors.  Later, tpc_vote
was added, which is a synchronous method. That would have been an
execllent opportunity to fix this, but noooooooooo.

I'd like the server vote logic to get simpler, and we also want to get rid
of serialnos, as it makes it hard to commit multiple transactions at
once on the client.  We can't get rid of serialnos now, because the
client needs to work with old servers.

Of course, nothing is easy. There was a special edge case where if the
client saw an unhandled conflict error, it would invalidate it's cache
for that object, allowing it to eventually recover when caches had
missed invalidations. This required a change to ClientStorage, which
meant that the server wouldn't work with older ZEO versions, which
meant that the server could only support protocol Z5, which in tern
affected protocol-negotiation tests....

  Commit: 2529c25b267b89db96e43703055305b0f77b8c71
      https://github.com/zopefoundation/ZEO/commit/2529c25b267b89db96e43703055305b0f77b8c71
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-02 (Sat, 02 Jul 2016)

  Changed paths:
    M src/ZEO/tests/ConnectionTests.py

  Log Message:
  -----------
  Removed a test that doesn't make sense any more

As it depends on getting serials from stpre or vote

  Commit: 8b4689d59cb14d52940d8e953a8d8ebce06154da
      https://github.com/zopefoundation/ZEO/commit/8b4689d59cb14d52940d8e953a8d8ebce06154da
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-02 (Sat, 02 Jul 2016)

  Changed paths:
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py
    M src/ZEO/TransactionBuffer.py
    M src/ZEO/tests/testZEO.py
    M src/ZEO/tests/testZEO2.py

  Log Message:
  -----------
  provide IMultiCommitStorage

  Commit: 02bb6611597b603b5bf485803d33fe82141c5200
      https://github.com/zopefoundation/ZEO/commit/02bb6611597b603b5bf485803d33fe82141c5200
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-05 (Tue, 05 Jul 2016)

  Changed paths:
    M .travis.yml
    A doc/README.rst
    R doc/zeo.txt
    M setup.py
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py
    M src/ZEO/__init__.py
    R src/ZEO/acceptor.py
    A src/ZEO/asyncio/README.rst
    M src/ZEO/asyncio/client.py
    A src/ZEO/asyncio/mtacceptor.py
    M src/ZEO/asyncio/server.py
    M src/ZEO/asyncio/testing.py
    M src/ZEO/component.xml
    M src/ZEO/runzeo.py
    M src/ZEO/schema.xml
    A src/ZEO/server.xml
    M src/ZEO/tests/CommitLockTests.py
    M src/ZEO/tests/ConnectionTests.py
    A src/ZEO/tests/client.pem
    A src/ZEO/tests/client_key.pem
    M src/ZEO/tests/dynamic_server_ports.test
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/new_addr.test
    A src/ZEO/tests/server.pem
    A src/ZEO/tests/server_key.pem
    A src/ZEO/tests/serverpw.pem
    A src/ZEO/tests/serverpw_key.pem
    M src/ZEO/tests/testConfig.py
    M src/ZEO/tests/testConnection.py
    M src/ZEO/tests/testZEO.py
    A src/ZEO/tests/testssl.py
    A src/ZEO/zconfig.py
    M tox.ini

  Log Message:
  -----------
  Merge branch 'ssl' into client-side-conflict-resolution

Conflicts:
	src/ZEO/ClientStorage.py

  Commit: 54c31e71bfa51085d6f093d6060bc901c5cdf555
      https://github.com/zopefoundation/ZEO/commit/54c31e71bfa51085d6f093d6060bc901c5cdf555
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-06 (Wed, 06 Jul 2016)

  Changed paths:
    M setup.py

  Log Message:
  -----------
  require newer ZODB needed for tests to pass

  Commit: 3d886d426243655b9f5a2528636e42b5c7662c19
      https://github.com/zopefoundation/ZEO/commit/3d886d426243655b9f5a2528636e42b5c7662c19
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-06 (Wed, 06 Jul 2016)

  Changed paths:
    M src/ZEO/asyncio/base.py

  Log Message:
  -----------
  Try to make sure TCP_NODELAY is set on INET sockets.

This is a nasty work around for: https://bugs.python.org/issue27456

It can't always be applied.  For example, it can't be applied to Linux
SSL sockets using the asyncio-provided loop.

  Commit: b8711838912041078cb0bd1706e57b5c4b678c9e
      https://github.com/zopefoundation/ZEO/commit/b8711838912041078cb0bd1706e57b5c4b678c9e
  Author: Jim Fulton <jim at jimfulton.info>
  Date:   2016-07-06 (Wed, 06 Jul 2016)

  Changed paths:
    M setup.py
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py
    M src/ZEO/TransactionBuffer.py
    M src/ZEO/asyncio/server.py
    M src/ZEO/asyncio/tests.py
    M src/ZEO/cache.py
    M src/ZEO/interfaces.py
    M src/ZEO/tests/ConnectionTests.py
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/protocols.test
    M src/ZEO/tests/testZEO.py
    M src/ZEO/tests/testZEO2.py

  Log Message:
  -----------
  Merge pull request #35 from zopefoundation/implement-IMultiCommitStorage

Implement iMultiCommitStorage

  Commit: 2f0021335aabb9501a57b69e1c7ac5266de05624
      https://github.com/zopefoundation/ZEO/commit/2f0021335aabb9501a57b69e1c7ac5266de05624
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-06 (Wed, 06 Jul 2016)

  Changed paths:
    M setup.py
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py
    M src/ZEO/TransactionBuffer.py
    M src/ZEO/asyncio/server.py
    M src/ZEO/asyncio/tests.py
    M src/ZEO/cache.py
    M src/ZEO/interfaces.py
    M src/ZEO/tests/ConnectionTests.py
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/protocols.test
    M src/ZEO/tests/testZEO.py
    M src/ZEO/tests/testZEO2.py

  Log Message:
  -----------
  Merge remote-tracking branch 'origin/asyncio' into ssl

Conflicts:
	src/ZEO/ClientStorage.py

  Commit: 22615fb463cd57979a411cc636f30a7965886be4
      https://github.com/zopefoundation/ZEO/commit/22615fb463cd57979a411cc636f30a7965886be4
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-06 (Wed, 06 Jul 2016)

  Changed paths:
    M src/ZEO/StorageServer.py

  Log Message:
  -----------
  removed some unused code

  Commit: accd8c023b4c0aedbbd5d16faf63618095588351
      https://github.com/zopefoundation/ZEO/commit/accd8c023b4c0aedbbd5d16faf63618095588351
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-06 (Wed, 06 Jul 2016)

  Changed paths:
    M src/ZEO/StorageServer.py
    A src/ZEO/tests/test_client_side_conflict_resolution.py
    A src/ZEO/tests/utils.py

  Log Message:
  -----------
  server side

  Commit: dabcb601343869fabd1578ccef1b45e904e61a01
      https://github.com/zopefoundation/ZEO/commit/dabcb601343869fabd1578ccef1b45e904e61a01
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-06 (Wed, 06 Jul 2016)

  Changed paths:
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py
    M src/ZEO/TransactionBuffer.py
    M src/ZEO/runzeo.py
    M src/ZEO/server.xml
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/test_client_side_conflict_resolution.py

  Log Message:
  -----------
  client side and server config and tweaks

  Commit: 4cdbf8b15796cfd2757e1737b63b55f412ba83d1
      https://github.com/zopefoundation/ZEO/commit/4cdbf8b15796cfd2757e1737b63b55f412ba83d1
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-06 (Wed, 06 Jul 2016)

  Changed paths:
    M src/ZEO/tests/CommitLockTests.py
    M src/ZEO/tests/ConnectionTests.py
    M src/ZEO/tests/testConversionSupport.py
    M src/ZEO/tests/testZEO.py

  Log Message:
  -----------
  fixed tests

  Commit: 92e49851c8f430c6ff72b4fb4a7e24e0ef391de3
      https://github.com/zopefoundation/ZEO/commit/92e49851c8f430c6ff72b4fb4a7e24e0ef391de3
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/testZEO.py

  Log Message:
  -----------
  run pre-defined conflict-resolution tests with client-side conflict resolution

  Commit: af1e20040a4609510200eb2f204f74aa9bdd09ae
      https://github.com/zopefoundation/ZEO/commit/af1e20040a4609510200eb2f204f74aa9bdd09ae
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M src/ZEO/asyncio/base.py

  Log Message:
  -----------
  Cleaner fix.

I'd missed that you could get a transport's socket using get_extra_info.

  Commit: 1eb086dae13b17304f5d4b0614872be3bb4aa447
      https://github.com/zopefoundation/ZEO/commit/1eb086dae13b17304f5d4b0614872be3bb4aa447
  Author: Jim Fulton <jim at jimfulton.info>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M .travis.yml
    A doc/README.rst
    R doc/zeo.txt
    M setup.py
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py
    M src/ZEO/__init__.py
    R src/ZEO/acceptor.py
    A src/ZEO/asyncio/README.rst
    M src/ZEO/asyncio/client.py
    A src/ZEO/asyncio/mtacceptor.py
    M src/ZEO/asyncio/server.py
    M src/ZEO/asyncio/testing.py
    M src/ZEO/component.xml
    M src/ZEO/runzeo.py
    M src/ZEO/schema.xml
    A src/ZEO/server.xml
    M src/ZEO/tests/CommitLockTests.py
    M src/ZEO/tests/ConnectionTests.py
    A src/ZEO/tests/client.pem
    A src/ZEO/tests/client_key.pem
    M src/ZEO/tests/dynamic_server_ports.test
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/new_addr.test
    A src/ZEO/tests/server.pem
    A src/ZEO/tests/server_key.pem
    A src/ZEO/tests/serverpw.pem
    A src/ZEO/tests/serverpw_key.pem
    M src/ZEO/tests/testConfig.py
    M src/ZEO/tests/testConnection.py
    M src/ZEO/tests/testZEO.py
    A src/ZEO/tests/testssl.py
    A src/ZEO/zconfig.py
    M tox.ini

  Log Message:
  -----------
  Merge pull request #32 from zopefoundation/ssl

Add SSL support +

It's sad no reviewer, but I suppose it can be reviewed even after a merge.

  Commit: 5a5320e625fc6899fb3cfcf8d986c13f54848c94
      https://github.com/zopefoundation/ZEO/commit/5a5320e625fc6899fb3cfcf8d986c13f54848c94
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M setup.py
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py

  Log Message:
  -----------
  Merge remote-tracking branch 'origin/asyncio' into client-side-conflict-resolution

Conflicts:
	src/ZEO/ClientStorage.py
	src/ZEO/StorageServer.py

  Commit: 10d7f9ffa115ece8d94e51bbddc601a8d670a69e
      https://github.com/zopefoundation/ZEO/commit/10d7f9ffa115ece8d94e51bbddc601a8d670a69e
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M src/ZEO/tests/forker.py

  Log Message:
  -----------
  fixed a waitpid call (unrelated to rest of branch, sorry)

  Commit: 70f070e80f5290cf9ba28dc90d96bc27579b98c7
      https://github.com/zopefoundation/ZEO/commit/70f070e80f5290cf9ba28dc90d96bc27579b98c7
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M src/ZEO/asyncio/testing.py

  Log Message:
  -----------
  need new transport extra

I'm losing my love of stubs. :(

  Commit: fc41070f655a2e77e7e03857eaf1a370a319c18c
      https://github.com/zopefoundation/ZEO/commit/fc41070f655a2e77e7e03857eaf1a370a319c18c
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M setup.py
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py
    M src/ZEO/TransactionBuffer.py
    M src/ZEO/asyncio/server.py
    M src/ZEO/asyncio/tests.py
    M src/ZEO/cache.py
    M src/ZEO/interfaces.py
    M src/ZEO/tests/ConnectionTests.py
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/protocols.test
    M src/ZEO/tests/testZEO.py
    M src/ZEO/tests/testZEO2.py

  Log Message:
  -----------
  Merge remote-tracking branch 'origin/asyncio' into TCP_NODELAY

  Commit: a0fa2fe71d22bc6af2c9b93bfe15847be06d0fe1
      https://github.com/zopefoundation/ZEO/commit/a0fa2fe71d22bc6af2c9b93bfe15847be06d0fe1
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M src/ZEO/tests/InvalidationTests.py

  Log Message:
  -----------
  Work around bugs in DB root-object initialization

See: https://github.com/zopefoundation/ZODB/issues/84

  Commit: bff1a14a161deb4a1b53a8d2e48643baddf5dae7
      https://github.com/zopefoundation/ZEO/commit/bff1a14a161deb4a1b53a8d2e48643baddf5dae7
  Author: Jim Fulton <jim at jimfulton.info>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M src/ZEO/asyncio/base.py
    M src/ZEO/asyncio/testing.py
    M src/ZEO/tests/InvalidationTests.py

  Log Message:
  -----------
  Merge pull request #37 from zopefoundation/TCP_NODELAY

TCP_NODELAY

  Commit: e5653d437904db4ab5809cef5257541677800c89
      https://github.com/zopefoundation/ZEO/commit/e5653d437904db4ab5809cef5257541677800c89
  Author: Jim Fulton <jim at jimfulton.info>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py
    M src/ZEO/TransactionBuffer.py
    M src/ZEO/runzeo.py
    M src/ZEO/server.xml
    M src/ZEO/tests/CommitLockTests.py
    M src/ZEO/tests/ConnectionTests.py
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/testConversionSupport.py
    M src/ZEO/tests/testZEO.py
    A src/ZEO/tests/test_client_side_conflict_resolution.py
    A src/ZEO/tests/utils.py

  Log Message:
  -----------
  Merge pull request #38 from zopefoundation/client-side-conflict-resolution

Client side conflict resolution

  Commit: e8bec203c38430de3809730901e2698595be6153
      https://github.com/zopefoundation/ZEO/commit/e8bec203c38430de3809730901e2698595be6153
  Author: Jim Fulton <jim at zope.com>
  Date:   2016-07-07 (Thu, 07 Jul 2016)

  Changed paths:
    M src/ZEO/ClientStorage.py
    M src/ZEO/StorageServer.py
    M src/ZEO/TransactionBuffer.py
    M src/ZEO/asyncio/base.py
    M src/ZEO/asyncio/server.py
    M src/ZEO/asyncio/testing.py
    M src/ZEO/asyncio/tests.py
    M src/ZEO/cache.py
    M src/ZEO/interfaces.py
    M src/ZEO/runzeo.py
    M src/ZEO/server.xml
    M src/ZEO/tests/CommitLockTests.py
    M src/ZEO/tests/ConnectionTests.py
    M src/ZEO/tests/InvalidationTests.py
    M src/ZEO/tests/forker.py
    M src/ZEO/tests/protocols.test
    M src/ZEO/tests/testConversionSupport.py
    M src/ZEO/tests/testZEO.py
    M src/ZEO/tests/testZEO2.py
    A src/ZEO/tests/test_client_side_conflict_resolution.py
    A src/ZEO/tests/utils.py

  Log Message:
  -----------
  Merge remote-tracking branch 'origin/asyncio' into uvloop

Conflicts:
	setup.py

Compare: https://github.com/zopefoundation/ZEO/compare/b76fea7d3dc2...e8bec203c384