[ZODB-Dev] RE: [Zope] Re: Squid headaches

Dieter Maurer dieter@handshake.de
Sun, 1 Jun 2003 22:30:59 +0200

Tim Peters wrote at 2003-5-31 14:58 -0400:
 > [Dieter Maurer]
 > > ...
 > > In the second packing phase (the so called copying phase),
 > > the packing thread helds the storage lock most of the time.
 > > This can make accessing the ZODB impossible by normal operations,
 > > especially because the "lock_release", "lock_acquire" probably
 > > does not succeed in giving the lock to a waiting thread.
 > That decision is made by the platform thread facilities:  the strongest that
 > can be said in general is that lock_release *offers* to yield to another
 > thread.  The conditions under which the platform accepts the offer vary a
 > lot across platforms.

In other (i.e. non-Zope) code, I saw code like:

	 sleep(0) # indicate, threads should switch, if necessary

Of course, "sleep(0)" may not force a thread switch on all platforms.

My lock time optimization holds the lock only while it decides
whether it has finished. It looks like:

                # Read the transaction record
                if len(h) < TRANS_HDR_LEN:
                    # DM: lock and try again
                    _commit_lock_acquire(); _lock_acquire(); locked= 1
                    h+= read(TRANS_HDR_LEN - len(h))
                    if len(h) < TRANS_HDR_LEN: break
                # DM: ensure the complete transaction can be read
                if not (packing or locked or _largeEnough(file,tend + 8)):
                    # it was not - lock
                    # when we get the lock, the transaction will be written completely
                    # this would not work with a concurrent pack;
                    # but hopefully, this cannot happen
                    _commit_lock_acquire(); _lock_acquire(); locked= 1
                    assert _largeEnough(file,tend+8)
	       # DM: we can release the locks now, as the currently
	       #     copied transaction is complete
	       if locked: locked= 0; _lock_release(); _commit_lock_release()

"_largeEnough(file,size)" checks that *file* has at least length *size*.

Because the locks are held only during a very short time, locking
out of other requests should be drastically reduced.