[ZODB-Dev] ZODB file never updated - import transaction fails...

[skip at pobox.com]

Matt Cowles has a DNS cache module with a patch for use with SpamBayes.  It
doesn't persist the data (just caches during the lifetime of the run).  I'm
trying to add persistence to that cache and wanting to use the same database
as the user selects for the main SpamBayes database, my first two targets
were anydbm and ZODB.  I'm having trouble with the ZODB setup.  It seemed to
work well at the start, but the database file is never updated when I rerun
my training script, so its not caching anything, and now everything in the
database has expired.  I've never used ZODB before.  The amount of code is
fairly small, so I'll just post it here in hopes someone can see what I've
done wrong.

When opening the database it selects based on the user-preferred filetype.
For "zodb" it executes:

    from ZODB import DB
    from ZODB.FileStorage import FileStorage
    self._zodb_storage = FileStorage(cachefile, read_only=False)
    self._DB = DB(self._zodb_storage, cache_size=10000)
    self._conn = self._DB.open()
    root = self._conn.root()
    self.caches = root.get("dnscache")
    if self.caches is None:
        # There is no classifier, so create one.
        from BTrees.OOBTree import OOBTree
        self.caches = root["dnscache"] = OOBTree()
        self.caches["A"] = {}
        self.caches["PTR"] = {}

That gives the cache a caches attribute that looks like the dictionary it
uses in the no persistence case.

When the cache is deleted, its __del__ method calls

    self.close()

which calls self._zodb_close():

  def _zodb_close(self):
      # Ensure that the db is saved before closing.  Alternatively, we
      # could abort any waiting transaction.  We need to do *something*
      # with it, though, or it will be still around after the db is
      # closed and cause problems.  For now, saving seems to make sense
      # (and we can always add abort methods if they are ever needed).
      self._zodb_store()

      # Do the closing.        
      self._DB.close()

which calls self._zodb_store():

  def _zodb_store(self):
      import transaction
      from ZODB.POSException import ConflictError
      from ZODB.POSException import TransactionFailedError

      try:
          transaction.commit()
      except ConflictError, msg:
          # We'll save it next time, or on close.  It'll be lost if we
          # hard-crash, but that's unlikely, and not a particularly big
          # deal.
          if options["globals", "verbose"]:
              print >> sys.stderr, "Conflict on commit.", msg
          transaction.abort()
      except TransactionFailedError, msg:
          # Saving isn't working.  Try to abort, but chances are that
          # restarting is needed.
          if options["globals", "verbose"]:
            print >> sys.stderr, "Store failed.  Need to restart.", msg
          transaction.abort()

which seems to fail.  I get this message in the log:

    Exception exceptions.ImportError: 'No module named transaction' in
    <bound method cache.__del__ of <spambayes.dnscache.cache instance at
    0x11994b8>> ignored

I can import transaction from the interpreter prompt just fine:

    % python
    Python 2.5b2 (trunk:50921, Jul 28 2006, 20:21:50) 
    [GCC 4.0.0 (Apple Computer, Inc. build 5026)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import transaction
    >>> transaction.__file__
    '/Users/skip/local/lib/python2.5/site-packages/transaction/__init__.pyc'
    >>> import ZODB
    >>> ZODB.__file__
    '/Users/skip/local/lib/python2.5/site-packages/ZODB/__init__.pyc'

I stuck a 

    print transaction.__file__

after the import in _zodb_close and ran dnscache as a main.  It prints out

    /Users/skip/local/lib/python2.5/site-packages/transaction/__init__.pyc

so I'm pretty sure I'm getting the same version of Python.  Any idea why it
would fail in one instance but not another?  The only thing I can think of
is that there are two ZODB databases opened at the same time in the real use
case, but only the one in the situation where I'm running the dnscache test
function.  Is there something about having two distinct ZODB database files
open I need to consider?

Thanks,

-- 
Skip Montanaro - skip at pobox.com - http://www.mojam.com/