[ZODB-Dev] weird conflicts with BTrees
Chris McDonough
chrism at plope.com
Wed Nov 30 23:11:16 UTC 2011
Various different strategies are explored in "faster":
http://agendaless.com/Members/tseaver/software/faster/ . But it still
uses BTrees.
Faster is a zope product that is the basis for repoze.session.
repoze.session explains itself here:
http://docs.repoze.org/session/internals.html
On Wed, 2011-11-30 at 06:12 -0500, Stephan Richter wrote:
> On Wednesday, November 30, 2011 11:44:20 AM Adam GROSZER wrote:
> > Hello,
> >
> > I thought BTrees have conflict resolution code.
> >
> > We have here a weird conflict here, the BTree in question is a
> > SessionDataContainer. The conflict seems to appear when one transaction
> > removes and the other adds an entry.
>
> I am working with Adam on this, so here is some more detail. First of all,
> this clearly does not happen all the time. It seems to happen only
> infrequently, which I think is in cases where the buckets of the BTree get
> reorganized due to a deletion sweep.
>
> I think the reason why this is a relatively common scenario is:
>
> 1. There is a lot of activity on the site at some point.
> 2. Then there is a prolonged period of inactivity, which causes a bunch of
> sessions to time out.
> 3. Then activity is starting again, causing the first transaction to do a large
> session removal sweep (which causes the BTree restructuring) and the next
> concurrent transaction to add a new session causing a conflict.
>
> At least for sessions, conflict resolution is not that difficult, because we are
> always dealing with unique keys and keys are never re-assigned to another
> object. So it effectively becomes a set conflict resolution issue.
>
> Also, I think that BTrees might be overkill for this problem, since the amount
> of active authenticated users on a system is usually small. And even if there
> is a large amount of users, the key->session mapping will always reside in the
> ZODB object cache (though there is some overhead to distributing new versions
> of the mapping due to the relatively high number of writes.)
>
> I am tempted to write a conflict resolving dictionary for this, but I would
> love hear whether someone has done something like this already.
>
> Regards,
> Stephan
More information about the ZODB-Dev
mailing list