[ZODB-Dev] Opening your own parallel transactions/database connections in Zope

[Dieter Maurer]

Christian Theune wrote at 2004-2-17 20:49 +0000:
> ...
>Now, in the loop of the thread I handle database connectivity in this
>way (copy of the code, a little bit simplified, there are some sleep()
>and application relevant lines missing):
>
>
>-----
>    def run2(self):
>        # try to get a zodb connection
>        work = None
>        app=Zope.bobo_application()
>        while 1:
>            if work:
>                work = 0
>                try:
>                    get_transaction().commit()
>                except:
>                    get_transaction().abort()
>                    check_app._p_jar.sync()
>                continue

This is a funny place for "commit/abort".

I think your code should look like:

	while 1:
	      self.doInTransaction(self.checkAndProcess)

With "doInTransaction" like:

	def doInTransaction(self,f):
	    while 1:
		  self.connection.sync()
		  try:
			f()
			get_transaction().commit()
		  except ConflictError:
			get_transaction().abort()
			# maybe retries controlled
			continue
		  except:
			get_transaction().abort()
			raise
		  else:	return


> ...
>I have the feeling that this code may disrupt some data in the ZODB

Your code may be funny but it should not disrupt ZODB data.
Funny code usually leads to more conflicts than necessary
and may (under rare conditions) lead to some inconsistencies
due to using out-dated data (this can even happen without
funny code -- the ZODB does not yet support true serializability).

> ...
>We had some data modified within Plone (and I ran the linkchecker
>every now and then) and suddenly some content was missing. The
>transactions weren't visible within the Undo-Log and running fsrecover
>on the Data.fs showed that exactly one hour was missing (clocks were set
>correctly.)

The ZODB helds locks only within internal processing.
Only transaction clients should be able to extend the time period
the locks are held. Than can happen when e.g. the connection
to a database hangs (due to network problems).
Huge commits (which may later be aborted in the second commit phase)
may also cause extended locking time.

>Additionally sometimes the whole Zope server hangs when running the
>linkchecker, but that may be a different problem as it could be through
>the lockyness of some calls I do when requesting data from your owner
>server...

In such case, I use a debugger (GDB under Linux) to see what
Zope is doing and what it is waiting for.

-- 
Dieter