[Zope3-Users] Re: zc.queue conflict question

[Gary Poster]

On May 20, 2007, at 6:26 AM, Chris Withers wrote:

> Gary Poster wrote:
>>> Why does adding to identical objects to a queue at the same time  
>>> result in a conflict? Surely they should both just get added in  
>>> an artbitary order?
>> Basically, the constraint allows for more powerful conflict  
>> resolution, or at least simpler code.
>
> Um, can you explain that? How is adding two dissimilar objects  
> different from adding two identical objects?
>
>> I'd certainly welcome a variation that removed the constraint,  
>> possibly in exchange for weaker conflict resolution, if you were  
>> willing to contribute it to zc.queue.
>
> Sure, but I'm still hazy on what the problem with adding two  
> identical objects is. Can you explain?

Sure.

The `pull` is actually the interesting part, and it becomes more so  
when you allow an arbitrary pull.  By asserting that you do not  
support having equal items in the queue at once, you can simply say  
that when you remove equal objects in the current state and the  
contemporary, conflicting state, it's a conflict error.  Ideally you  
don't enter another equal item in that queue again, or else in fact  
this is still an error-prone heuristic:

- start queue == [X];
- begin transactions A and B;
- B removes X and commits;
- transaction C adds X and Y and commits;
- transaction A removes X and tries to commit, and conflict  
resolution code believes that it is ok to remove the new X from  
transaction C because it looks like it was just an addition of Y).   
Commit succeeds, and should not.

If you don't assert that you can use equality to examine conflicts,  
then you have to come up with another heuristic.  Given that the  
conflict resolution code only gets three states to resolve, I don't  
know of a reliable one.

Therefore, zc.queue has a policy of assuming that it can use equality  
to distinguish items.  It's limiting, but the code can have a better  
confidence of doing the right thing.

Also, FWIW, this is policy I want: for my use cases, it would be  
possible to put in two items in a queue that handle the same issue.   
With the right equality code, this can be avoided with the policy the  
queue has now.

It's worth noting that putting persistent.Persistent objects (or non- 
persistent objects that refer to persistent.Persistent objects in  
code used for conflict resolution) into objects with conflict- 
resolving behavior such as BTrees or zc.queues will stymie conflict  
resolution code, because persistent.Persistent objects other than the  
conflicted one are not around.  This leads to a problem Jim and I  
recently discovered with zope.app.keyreference.persistent being used  
with intids.  I intend to write up some blurbs about this and other  
issues in the zc.queue docs (although they are pertinent to BTrees  
too) asap.  A collector issue would probably be in order too.

Gary