[ZODB-Dev] Re: ReadConflictError while re-indexing big fat in dex

[Tino Wildenhain]

Hi,

Am Mo, den 02.08.2004 schrieb sean.upton at uniontrib.com um 21:41:
> Casey Duncan wrote:
> > > I presume you mean index objects in batches with a commit 
> > in between?
> > > But that creates an inconsistent state surely?
> > 
> > That means some objects are reindexed and others aren't. 
> > Whether that is acceptable or not is up to you.
> 
> This also has a side-effect: storage bloat between packing. As you commit
> more frequently, the number of stale copies of indexes goes up, sometimes in
> ways that are quite surprising - or quite painful.  This also increases pack
> time, which is salt in the wounds.
> 
> > Another option is to use versions, which will lock the 
> > catalog indexes that are changed in the version and prevent 
> > any concurrent updates to it. This will prevent read conflict 
> > errors, but will cause version lock errors if other clients 
> > attempt to write.
> 
> Wow - a practical use case for versions!
> 
> Using versions as a lock isn't something I've thought to do before.  So you
> have two processes writing to the indexes - A and B - A is more important
> and starts first, then locks the indexes by entering a version then touching
> each of them; B runs, tries to update, and fails with a lock error.  Does
> this sound right?  If so, at least this avoids both the RCE and the bloat
> asssociated with the alternative of increasing number/frequency of commits?
> 
Otoh, this is practical a write lock.
Isnt it possible to just use the last commited
historic version of the catalog index
(See OFS.History)? Could be half (if not all)
of the MVCC...

Regards
Tino