[ZODB-Dev] BerkeleyStorage is gone - what now???

[Harald Koschinski]

Hi,

some minutes ago I realised, that support for BDB is gone in ZODB3 
3.3.2. This is really hard for me, because I decided to use BDB as my 
main storage. :-((

I don't have a problem without BDB if there is a comparable storage 
available - but I can not see such storage. I need to store some 100's 
GB in a distributed way (ZEO). BDB is doing this in a nearly perfect way 
- fast, reliable and a small administration overhead.

The alternatives:

1. FileStorage: Needs to much memory for big databases.
2. APE: No ZEO support, Alpha quality, no versions
3. Directory-Storage: Only 1 Maintainer and a "problematic technical 
design" (Filesystem as databasefile??), no versions


I can not understand the reasons for stopping BDB-Support at 
http://zope.org/Wikis/ZODB/BerkeleyStorageDocs/BDBStorage.html


    * The usual range of bug reports against an immature implementation.
      For example, reports of autopack not working, and of manual
      packing never completing -- even reports of packing /increasing/
      storage size.

immature?? autopacking works perfectly!!

    * A way to prevent unbounded growth of Berkeley log files, without
      shutting down the app. This is apparently complicated due to that
      the ZODB BDB storage implementations disable Berkeley DB's native
      locking mechanisms (a large transaction could require more locks
      than bsddb can provide).

a cronjob is running moving the logfiles, I tested with 300GB 
transactions without any problems !!

    * The maze of incompatible Berkeley DB distributions on different
      platforms. A striking example of this problem was that the ZODB
      Berkeley tests died with segfaults if the platform had a more
      recent version of Berkeley DB than the ZODB code was developed
      against. There was a long history of version-specific ZODB BDB
      test failures before that.

different versions of different software have to adjust - that's part of 
softwaredevelopement.

    * Tuning Berkeley DB for performance appeared complicated,
      difficult, and necessary.

indeed - I had to set some parameters in DB_CONFIG (set_lg_dir 
/dumpdir/archive_logs/bdb/dms, set_tmp_dir /tmp/bdb, set_lg_max   
419430400,  set_lg_bsize 104857600,  set_flags DB_TXN_NOSYNC) - is this 
a problem ???


I really don't understand the descision - specially because there are no 
alternatives in sight.

Any suggestions?

Harald





 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.zope.org/pipermail/zodb-dev/attachments/20040804/323a4da0/attachment.htm