[ZODB-Dev] Re: Database size increase on commit()

[Jürgen Kartnaller]

tsmiller wrote:
> 
> Tres Seaver wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> tsmiller wrote:
>>
>>> This is what my gut feeling was telling me.  Before I knew ANYTHING about
>>> the ZODB it seemed natural and easy to just use dictionaries as data
>>> structures for things such as accounts, books, dialogs, etc..   But it
>>> looks
>>> like the database saves the entire dictionary if anything in it changes. 
>>> Is
>>> this correct and is this why I need to use the PersistentMappings or
>>> BTrees?
>> Yes, you are correct.
>>
>>> Why would I use one over the other?  
>> Both obey the "mapping protocol" used by Python.  PersistentMapping
>> tries harder to work like a dict;  the various BTee variants tend to
>> scale better when the mapping gets large, at the cost of O(logN) lookup
>> behavior, vs. O(1) amortizedd for the PM.
>>
>>
>>
>> Tres.
>> - --
>> ===================================================================
>> Tres Seaver          +1 540-429-0999          tseaver at palladion.com
>> Palladion Software   "Excellence by Design"    http://palladion.com
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.6 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>>
>> iD8DBQFHRIso+gerLs4ltQ4RAhtNAKDbG3MzQXiG5cofIjmL23o/10Z80gCeMMGX
>> pRcV4PRFu7cg2Nmg1g8qSp8=
>> =H4hQ
>> -----END PGP SIGNATURE-----
>>
>> _______________________________________________
>> For more information about ZODB, see the ZODB Wiki:
>> http://www.zope.org/Wikis/ZODB/
>>
>> ZODB-Dev mailing list  -  ZODB-Dev at zope.org
>> http://mail.zope.org/mailman/listinfo/zodb-dev
>>
>>
> It is late. I am tired. I have been looking at this stuff since this
> morning.  If possible, would you take a quick look at this short program and
> see if you can  take a guess why it only saves the first record.  The other
> two are completely ignored.  All of the records are written to memory, but
> only the first comes back when I restart the program. Of course, it is
> probably something obvious.  Thanks. Tom
> 
> from ZODB import FileStorage, DB
> import BTrees.OOBTree
> 
> import persistent, transaction
> import time
> storage = FileStorage.FileStorage('books.fs')
> db = DB( storage)
> conn = db.open()
> dbRoot = conn.root()
> bTree = BTrees.OOBTree
> 
> class BookStore(persistent.Persistent):
>     def __init__(self):
>         if not dbRoot.has_key("books"):
>             print "creating books"
>             dbRoot['books'] = bTree.OOBTree()
>             dbRoot['books']['1'] = {}

Change the line above to :
   dbRoot['books']['1'] = bTree.OOBTree()

The reason why this doesn't work is that ZODB doesn't know about your 
changes in the dict. It knows about the changes in the btree so the 
first insert is commited because it creates a new entry in the btree.

>         
>     def addBook(self, key, book):
>         print "Adding a book ", key, book
>         try:
>             dbRoot['books']['1'][key] = book     
>             transaction.commit()            
>         except:
>             print "transaction did not commit"
> 
>         
> if __name__ == '__main__': 
> 
>     print "at the beginning"
>     store, books = dbRoot['books'].iteritems().next()
> ##    print "store  ", store, " books ", books
>     for key, book in books.items():
>         print key, book
>     
>     bookStore = BookStore()
>     bookStore.addBook('1-21', {'book-id' : "1-21", 'title':"The Red Pony"})    
>     bookStore.addBook('1-20', {'book-id' : "1-24", 'title':"The Grapes of
> Wrath"})
>     bookStore.addBook('1-56', {'book-id' : "1-56", 'title':"Python Rules"})
>     
> ##    print "at the end"
> ##    store, books = dbRoot['books'].iteritems().next()
> ####    print "store  ", store, " books ", books
> ##    for key, book in books.items():
> ##        print key, book
>     ##print dbRoot['books'].iteritems().next() ,
> type(dbRoot['books'].iteritems().next())
>     db.close()
> 
>