[ZODB-Dev] Re: bsddb3Storage CVS

[Toby Dickenson]

Following from an email conversation with Barry:

I wrote, in http://collector.zope.org/Zope/283

>bsddbStorage doesnt perform its own checkpointing; you have to use
>the external db_checkpoint tool.

>bsddbStorage always performs database recovery on startup, and I believe 
>that it is possible to corrupt the database if checkpointing overlaps
>with recovery.

>It would be nice if bsddbStorage would intermittantly perform its
>own checkpointing. Perhaps in another thread

Barry replies:

>+0.  Full.py does do two checkpoints on database close.  It might make
>sense to occasionally do additional checkpoints, either in another
>thread, or in response to some counter (i.e. X number of commits).
>I'd be up for discussing this on zodb-dev.

The time taken for database recovery is proportional to the amount of data 
written since the last-but-one checkpoint, so doing it *only* on close isnt 
really enough if you expect the database to be open for a long time.

Checkpointing isnt blindingly fast... I usually see it take roughly half a 
second, but sometimes longer.  Thats long enough that we dont want to block a 
ZEO server, so I think its worth doing the checkpointing in another thread, 
or another process.