[ZODB-Dev] Relstorage and over growing database.

[Juan A. Diaz]

2013/2/2 Shane Hathaway <shane at hathawaymix.org>:
> On 02/01/2013 09:08 PM, Juan A. Diaz wrote:
>>
>> Reading the some comments [0] in the code
>> (relstorage/adapters/schema.py) I could see that the object_ref
>> database is uses during the packing, then the question is, in a
>> history-preserving database there is something that we could do to
>> decrease the size of that table? could be safe truncate that table? We
>> want to move to a storage with history-free, but for now we are
>> looking some options and actions to perform on production without the
>
>
> object_ref is essentially a cache of object_state, and object_refs_added is
> a list of what's in that cache.  Therefore you can freely truncate
> object_ref as long as you also truncate object_refs_added.  Don't truncate
> them during packing, though.

Do you think that add one option in zodbpack to truncate this tables
after the pack could be a god idea?

>
>> When we realize that the size of the database was over growing we
>> start to make daily packs but now after read this comment I start to
>> think that that could be now part of the problem, could be? Normally
>> in a normal day the DB grows like 2.2GB, but after make a pack the DB
>> size is decrease clouse to 1.5GB or 2GB.
>
>
> If your database grows by 2.2 GB per day, it's not surprising that the
> database is 15 GB.  With drive and RAM sizes today, 15 GB doesn't sound like
> a problem to me... unless it's on a Raspberry Pi. :-)
>
> Shane
>

Yes, but after the pack the size of new objects that remain in the
database is just like 200MB.

Also 15GB as you say is not a real big database for this days, but we
are synchronizing our databases through a low bandwidth channel across
various datacenters and in some cases recover the database from a
failure in the sync process is real pain!. Do you think that is
possible to don't replicate that tables could be safe? There are other
tables that maybe we don't need replicate?

Cheers and many thanks for you time :)

nueces...