[ZODB-Dev] Re: Heavy size increasing of database after adding an object

[Manuel Markwort]

Sorry about my late reply, first I had to make several changes to my 
code, but now it works and enlarging of data.fs' size is not so horrible 
yet.

Thanks to all for your prompt help about my problem. All of you were 
right, I found out that the high enlarging has its origin in several 
wrong designed classes (because there were some dictionaries which are 
not of type PersistentMapping, thanks Tim). After changing the code I 
could decrease the increasing about 4/5.

Refering to Malcom (see below), is it possible to disable the undo? 
Because I'm adding about 1500 objects to the database in a short period 
of time and I don't really need the undo.

Thanks very much,
Manuel

Malcolm Cleaton wrote:

>On Wed, 01 Dec 2004 17:20:07 +0100, Manuel Markwort wrote:
>  
>
>>I adding my own ZClass Objects in the data.fs. After adding a new 
>>instance of those objects the database size increases heavily in 
>>opposition of the real object size. Before I add the instance to the 
>>database it's size is 26.6MB, after commiting the transaction the size 
>>is about 26.9MB. After packing the database the size is 26.6MB again. 
>>What happened in the background? It is possible to avoid this high 
>>increasing of the database without packing after adding the object?
>>    
>>
>
>The temporary size increase of the database is caused by all the existing
>objects that you've changed. Whenever you change an object, a new copy is
>created in your transaction, and the old copy hangs around in the Data.fs
>until you pack.
>
>"But wait!" I hear you cry, "I didn't change anything, I just added my new
>object!" Well, you probably added it to a Folder, and as a result the
>Folder object has been changed - it now has a different list of children.
>Depending on the details, other objects may have been changed too.
>
>As an example, I added a couple of empty Python Scripts to my database,
>one in a folder with not many items, and one in a very busy folder. I then
>added a new page, which adds a few objects and also touches various
>catalog indexes etc.
>
>Script in small folder: +6700 bytes
>Script in large folder: +154,907 bytes
>New page: +657,405 bytes
>
>Total: +819,012 bytes
>After packing: +57,728 bytes
>
>In short, around 800Kb of Data.fs size increase, 93% of which was old
>versions of changed objects, and only 7% of which was the new content I
>added and the actual size increase of existing objects.
>
>If you need transaction undo support, you can't avoid this, although you
>can minimise the additional space needed by designing your object
>relationships carefully, so that you don't often need new versions of
>large objects.
>
>If you can do without undo, then theoretically a Storage implementation
>could avoid this growth, but I don't know if there's anything
>production-ready which does this.
>
>I believe the traditional approach is to ensure you have plenty of space,
>and to pack regularly.
>
>Thanks,
>Malcolm.
>
>  
>