[Zope] long running methods

[Jürgen Herrmann]

hi there!

as our zope2 zodb grows we experience more and more conflicts 
with long running transactions. so i'm going to rewrite these
methods to commit transactions in batches - here my attempt
to tackle this problem. the idea is to get a list of objects
to be modified in this transaction, break it down to batches
and commit after each batch was processed. to avoid zope's 
retry-on-conflict machinery and still have a chance to retry 
per batch transaction commits, the iteration over one batch
is guarded by a try:except ConflictError: and a retry logic.

i wrote some more or less pseudo code to make this clear.

def myLongRunningMethod(self):

  BATCH_SIZE = 10
  MAX_CONFLICTS = 3

  work_items = [some, work, to , do, ...]
  # list of persistent objects to be modified in this method

  idx = 0
  while idx < len(work_items):
    conflicts = 0
    try:
      my_batch = work_items[idx, idx+BATCH_SIZE]
      for work_item in my_batch:
        do_some_work(work_item)
      transaction.commit()
    except ConflicError:
      conflicts += 1
      if conflicts > MAX_CONFLICTS:
        raise
    else:
      idx += BATCH_SIZE

does this sound like a reasonable approach?

jürgen
-- 
>> XLhost.de - eXperts in Linux hosting ® <<

XLhost.de GmbH
Jürgen Herrmann, Geschäftsführer
Boelckestrasse 21, 93051 Regensburg, Germany

Geschäftsführer: Volker Geith, Jürgen Herrmann
Registriert unter: HRB9918
Umsatzsteuer-Identifikationsnummer: DE245931218

Fon:  +49 (0)800 XLHOSTDE [0800 95467833]
Fax:  +49 (0)800 95467830

WEB:  http://www.XLhost.de
IRC:  #XLhost at irc.quakenet.org