[ZODB-Dev] Re: Heavy size increasing of database after adding an object

[Malcolm Cleaton]

On Wed, 01 Dec 2004 17:20:07 +0100, Manuel Markwort wrote:
> I adding my own ZClass Objects in the data.fs. After adding a new 
> instance of those objects the database size increases heavily in 
> opposition of the real object size. Before I add the instance to the 
> database it's size is 26.6MB, after commiting the transaction the size 
> is about 26.9MB. After packing the database the size is 26.6MB again. 
> What happened in the background? It is possible to avoid this high 
> increasing of the database without packing after adding the object?

The temporary size increase of the database is caused by all the existing
objects that you've changed. Whenever you change an object, a new copy is
created in your transaction, and the old copy hangs around in the Data.fs
until you pack.

"But wait!" I hear you cry, "I didn't change anything, I just added my new
object!" Well, you probably added it to a Folder, and as a result the
Folder object has been changed - it now has a different list of children.
Depending on the details, other objects may have been changed too.

As an example, I added a couple of empty Python Scripts to my database,
one in a folder with not many items, and one in a very busy folder. I then
added a new page, which adds a few objects and also touches various
catalog indexes etc.

Script in small folder: +6700 bytes
Script in large folder: +154,907 bytes
New page: +657,405 bytes

Total: +819,012 bytes
After packing: +57,728 bytes

In short, around 800Kb of Data.fs size increase, 93% of which was old
versions of changed objects, and only 7% of which was the new content I
added and the actual size increase of existing objects.

If you need transaction undo support, you can't avoid this, although you
can minimise the additional space needed by designing your object
relationships carefully, so that you don't often need new versions of
large objects.

If you can do without undo, then theoretically a Storage implementation
could avoid this growth, but I don't know if there's anything
production-ready which does this.

I believe the traditional approach is to ensure you have plenty of space,
and to pack regularly.

Thanks,
Malcolm.

-- 

    [] j a m k i t
      web solutions for charities

         malcolm cleaton
T:  020 7549 0520
F:  020 7490 1152
M:  07986 563852
W: www.jamkit.com