[ZODB-Dev] maintenance of a BerkeleyDB

[Joseph Wayne Norton]

Andrew -

The support section of the sleepy cat website seems to have the most
information regarding how to maintain a berkeley database.

I also put together a script based on sleepy cat website information
for this purpose as well.  This script is only a few days old so there
might still be some issues.

 - you will need to modify the file paths, etc. to match your
environment

 - this script should be placed in a crontable for the backup
frequency that you desire

 - if you need to recover, simply extract the files from cpio archive.

 - this script assumes you have a given directory structure so you
might need to modify as necessary.

If you have any suggestions on how to improve, please let me know.

regards,

- joe n.

#!/bin/sh
ROTATE=/opt/arseed/tfs-lib/bdb/db-bdb-3.3/bin/db_archive
TFSVar=/u01/arseed/tfs-var/ios04-zeo01-2.3

cd $TFSVar

#
# default variables
#
MAXDAYS=1
MAXSIZE=104857600

for i in \
        I00/prd/shop_session \
        I03/prd/acl_users_session \
  do
    #
    # loop variables
    #
    DATE=`date +%Y.%m.%d.%H.%M`
    PREFIX=`hostname`
    PREFIX="$PREFIX$BASENAME"

    BSDDB="$TFSVar/var/vtfs/$i"
    BSDDBCPIO="$BSDDB/archive/$PREFIX.$DATE.cpio"
    BSDDBBASE="$BSDDB/archive/baseline.cpio"

    #
    # check for target directory
    #
    if [ ! -f "$BSDDB/data/DB_CONFIG" ]; then
        continue
    fi
    if [ ! -d "$BSDDB/tmp" ]; then
        continue
    fi
    if [ ! -d "$BSDDB/archive" ]; then
        continue
    fi

    #
    # change working dir
    #
    cd "$BSDDB/tmp"

    #
    # remove log files older than X days
    #
    find "$BSDDB/archive" -name "$PREFIX*" -ctime +$MAXDAYS -exec rm -f {} \; -print

    # archive data
    if [ ! -f "$BSDDBCPIO" ]; then
        # archive DB_CONFIG
        echo "$BSDDB/data/DB_CONFIG"  | cpio -m -co -O $BSDDBCPIO
        # archive data
        $ROTATE -s -a -h $BSDDB/data | cpio -m -co -O $BSDDBCPIO -A
        # archive logs
        $ROTATE -l -a -h $BSDDB/data | cpio -m -co -O $BSDDBCPIO -A
        # delete old logs
        $ROTATE    -a -h $BSDDB/data | /usr/local/bin/perl -pe 'chop, unlink($_);';

        if [ ! -f "$BSDDBBASE" ]; then
            mv $BSDDBCPIO $BSDDBBASE
        else
            CMP=`cmp $BSDDBBASE $BSDDBCPIO`
            if [ "$CMP" ]; then
                cp -f -p $BSDDBCPIO $BSDDBBASE
            else
                rm -f $BSDDBCPIO
            fi
        fi
    fi

    #
    # remove any log files that put the directory over the quota/size limit
    #
    FIN=0
    while [ $FIN = 0 ]
    do
        # get the size of PREFIX* files in the log directory
        SIZE=`ls -l $BSDDB/archive/$PREFIX* | awk 'BEGIN { total=0 } { total+=$5 } END { print total }'`
        # is the directory over the limit??
        if [ $SIZE -gt $MAXSIZE ]; then
            TODELETE=`ls -tr $BSDDB/archive/$PREFIX* | head -1`
            rm -f $TODELETE
        else
            # we're done
            FIN=1
        fi
    done
done

exit