<div>i would think it would be more useful immediately to account for the rate of growth rather than trying to account for all objects in the zodb. i've attached a quick script you can use to get some introspection on the last few transactions of a large data.fs file. it operates in read only mode, just pass the Data.fs file as an argument. For each of the last 20 (default) transactions it will report the objects (by class name) in the transaction and their sizes aggregated by class. hopefully based on the classes being written you can identify where the large writes are happening. </div>
<div><br></div><div>cheers,</div><div><br></div><div>kapil</div><div><br></div><div><br><br><div class="gmail_quote">On Mon, Dec 7, 2009 at 3:08 PM, Jose Benito Gonzalez Lopez <span dir="ltr"><<a href="mailto:jose.benito.gonzalez@cern.ch" target="_blank">jose.benito.gonzalez@cern.ch</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
Actually, we have two different versions (current and beta) of the software,<br>
working against the same ZODB. We are just developing new features<br>
on top of the beta version.<br>
Fearing that it was due to some changes done by a programmer<br>
we have disabled the "beta version" but the growing of the DB is still<br>
there...and since our last email we've got 5GB more (71GB now) .... We<br>
are a bit worried<br>
since the disk space is 102GB and the application is widely used here<br>
at CERN and<br>
for outsiders too.<br>
<br>
So, up-to-now we are afraid that it is not a problem with any<br>
modifications in the code....and<br>
no clue about it.<br>
We are trying to pack to see the difference with the data.fs file.<br>
<br>
Jim, I am still running your script "class_stats.py".....and it seems it<br>
is going to take a while yet....<br>
<br>
Thank you so much for your help. More ideas appreciated!<br>
<br>
Best regards,<br>
<font color="#888888">Jose<br>
</font><div><div></div><div><br>
<br>
<br>
<br>
Alan Runyan wrote:<br>
>> I'd just like to add that there's some changes that can be related to this:<br>
>><br>
>> - we had some classes inheriting from Persistent that now inherit from<br>
>> something else as well (but no extra arguments are being added, AFAIK);<br>
>> - we added some zope.interface definitions to some Persistent classes;<br>
>><br>
>> maybe this causes some kind of behavior that we were not aware of?<br>
>><br>
><br>
> I doubt it. Thousnads of people are doing this and do not report the same<br>
> behavior.<br>
><br>
> What is more likely is that a programmer is changing a persistent object<br>
> very often. One of the downsides of the ZODB is that it is so transparent<br>
> it is possible to unwillingly make database changes.<br>
><br>
> A design pattern for RDBMS is to have 2 pools. READ pool and WRITE pool.<br>
> Often the READ pool comes from some replica and WRITE is to the master.<br>
> I'm unsure this pattern would work for ZODB. I know Malthe was thinking<br>
> about this but unsure if he had anything concrete.<br>
><br>
> cheers<br>
> alan<br>
><br>
<br>
_______________________________________________<br>
For more information about ZODB, see the ZODB Wiki:<br>
<a href="http://www.zope.org/Wikis/ZODB/" target="_blank">http://www.zope.org/Wikis/ZODB/</a><br>
<br>
ZODB-Dev mailing list - <a href="mailto:ZODB-Dev@zope.org" target="_blank">ZODB-Dev@zope.org</a><br>
<a href="https://mail.zope.org/mailman/listinfo/zodb-dev" target="_blank">https://mail.zope.org/mailman/listinfo/zodb-dev</a><br>
</div></div></blockquote></div><br></div>