[ZODB-Dev] StorageServer's waiting list

[Christian Theune]

On 11/09/2009 08:06 PM, Jim Fulton wrote:
>>
>> [...]
>>
>> Z1a will not be able to issue another commit, those are blocked locally
>> by ClientStorage but pure reading transactions will go through.
>>
>> Z1b will be fine because ZEO1 knows about the ongoing commit and puts
>> Z1b into the waiting list when trying to vote, allowing other reads from
>> that connection to go through.
>>
>> However, when Z2 tries to commit, it starts filling the commit log on
>> ZEO2. ZEO2 doesn't know about the ongoing commit on the upstream ZEO and
>> will allow the vote phase to go upstream. However, know the commit from
>> Z2 gets stuck because it is put in the waiting list on the upstream ZEO
>> while ZEO2 thinks it was able to proceed.
> 
> Right, the vote call from ZEO2 blocks until the server is able to complete
> the in process transaction.

Ok, I think I got it now. So for that (small?) period in time all ZEO
clients for ZEO2 that cause a vote will also be blocked from any other
calls for that time.  Whereas ZEO clients from ZEO1 will be able to vote
but still continue to do other calls, like load.

So the behaviour of clients of ZEO1 differs a bit from the behaviour of
clients of ZEO2. This only happens when the commit log was already
transferred to the top-most ZEO the time everyone will stuck depends on
how fast it can process the commit log into the storage and complete the
commit (+time for conflict resolution?).

>> This will cause Z2 to completely become stuck and not benefit from the
>> waiting list on ZEO2.
>>
>> Sorry for the bloated example, I think it's the smallest way to explain it.
>>
>> Am I misunderstanding how the waiting list works?
> 
> No, but exactly the same thing would happen without fan out.
> 
> Consider a simpler example. There are 2 ZEO clients, C1 and C2, of a server, S.
> 
> C1 calls tpc_vote on S (after calling tpc_begin and making some store
> calls.) It's vote call
> proceeds because there weren't other transaction committing on S.
> 
> Now C2 calls tpc_vote. It will block, until C1 calls tpc_finish or
> tpc_abort.  It is just as
> blocked as Z2 is in your fan-out example above.

Well, difference being that C2 will still be able to perform other calls
in parallel over its connection as it's put in the waiting list on S.

What I'm trying to point out in my above scenario is that ZEO2 will
block all calls as it's not in the waiting list of ZEO2 but on ZEO.

Christian


-- 
Christian Theune · ct at gocept.com
gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany
http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1
Zope and Plone consulting and development