[ZODB-Dev] ZODB file never updated - import transaction fails...

[Dieter.Maurer at Haufe.de]

skip at pobox.com wrote at 2006-7-30 11:06 -0500:
> ...
>    try:
>        import dnscache
>        cache = dnscache.cache(cachefile=os.path.expanduser("~/.dnscache"))
>        cache.printStatsAtEnd = True
>    except (IOError, ImportError):
>        cache = None
>    else:
>        import atexit
>        atexit.register(cache.close)
>        print "calling", cache.close, "at exit"
>
>It dutifully prints "calling <bound method cache.close of
><spambayes.dnscache.cache instance at 0x119a4b8>> at exit" at startup.  I
>added a print to the close method.  It prints "closing zodb" at the right
>time.  There is no longer any complaint about importing the transaction
>module.  As far as I can tell, it should be working.  However, when I look
>at the timestamps on the various files, it appears only the .dnscache.index
>file has been updated:
>
>    % ls -ltr ~/.dnscache*
>    -rw-rw-r--   1 skip  staff  44547 Jul 23 09:06 /Users/skip/.dnscache.old
>    -rw-rw-r--   1 skip  staff      0 Jul 23 09:07 /Users/skip/.dnscache.tmp
>    -rw-rw-r--   1 skip  staff  44383 Jul 29 08:22 /Users/skip/.dnscache
>    -rw-rw-r--   1 skip  staff    293 Jul 29 18:13 /Users/skip/.dnscache.index
>
>(It's about 18:20 here right now.)
>
>So this code appears to be getting called and executes to completion:
>
>    def _zodb_store(self):
>        import transaction
>        from ZODB.POSException import ConflictError
>        from ZODB.POSException import TransactionFailedError
>
>        try:
>            transaction.commit()
>        except ConflictError, msg:
>            # We'll save it next time, or on close.  It'll be lost if we
>            # hard-crash, but that's unlikely, and not a particularly big
>            # deal.
>            if options["globals", "verbose"]:
>                print >> sys.stderr, "Conflict on commit.", msg
>            transaction.abort()
>        except TransactionFailedError, msg:
>            # Saving isn't working.  Try to abort, but chances are that
>            # restarting is needed.
>            if options["globals", "verbose"]:
>              print >> sys.stderr, "Store failed.  Need to restart.", msg
>            transaction.abort()
>
>    def _zodb_close(self):
>        # Ensure that the db is saved before closing.  Alternatively, we
>        # could abort any waiting transaction.  We need to do *something*
>        # with it, though, or it will be still around after the db is
>        # closed and cause problems.  For now, saving seems to make sense
>        # (and we can always add abort methods if they are ever needed).
>        self._zodb_store()
>
>        # Do the closing.        
>        self._DB.close()
>
>        # We don't make any use of the 'undo' capabilities of the
>        # FileStorage at the moment, so might as well pack the database
>        # each time it is closed, to save as much disk space as possible.
>        # Pack it up to where it was 'yesterday'.
>        # XXX What is the 'referencesf' parameter for pack()?  It doesn't
>        # XXX seem to do anything according to the source.
>        self._zodb_storage.pack(time.time()-60*60*24, None)
>        self._zodb_storage.close()
>
>        self._zodb_closed = True
>        if options["globals", "verbose"]:
>            print >> sys.stderr, 'Closed dnscache database'
>
>There are no prints about conflicts or failed stores.  No import errors, the
>final "Closed dnscache database" line is output.  Still, there is no update
>of the actual data on disk.  Recalling this code:
>
>    from ZODB import DB
>    from ZODB.FileStorage import FileStorage
>    self._zodb_storage = FileStorage(cachefile, read_only=False)
>    self._DB = DB(self._zodb_storage, cache_size=10000)
>    self._conn = self._DB.open()
>    root = self._conn.root()
>    self.caches = root.get("dnscache")
>    if self.caches is None:
>        # There is no classifier, so create one.
>        from BTrees.OOBTree import OOBTree
>        self.caches = root["dnscache"] = OOBTree()
>        self.caches["A"] = {}
>        self.caches["PTR"] = {}
>
>do I need to do something with self.caches or with the root object to
>actually flush the data to the file?

In principle, this is not necessary.

What you see is very strange.

I could only imagine that the transaction you commit is not the
transaction the modifications are registered with.

This could, e.g., happen when you modify the objects in a thread
different from where the transaction is committed (the default
transaction manager manages a transaction per thread).
However, you code does not give a hint that this is indeed
the problem.



-- 
Viele Grüße
Dieter

Tel: 0681-761 8751 bzw. 06894-870 177