Calling obj._p_deactivate() didn't help either....<br><br>I made a few simple tests:<br><br>if I run a empty loop (with just a pass statement), its fine. However If I something as simple as printing an attribute of the object, there goes my memory....
<br><br>thanks,<br><br>Flávio<br><br><div class="gmail_quote">On Jan 17, 2008 1:52 PM, Tres Seaver <<a href="mailto:tseaver@palladion.com">tseaver@palladion.com</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
-----BEGIN PGP SIGNED MESSAGE-----<br>Hash: SHA1<br><div class="Ih2E3d"><br>Flavio Coelho wrote:<br>> Hi,<br>><br>> I wrote a simple script to move data from a ZODB database to sqlite.<br>> My data is in a OOBTree, I chose this because supposedly you can bring the
<br>> buckets to memory one at a time....<br>><br>> So what I am doing is basically this: I iterate over my objects and write<br>> them one-by-one to the other db.<br>> for k,u in user_root['userdb'].items(): # I have tried iteritems() here too
<br>> but the results are the same....<br>> # write the data to sqlite<br>><br>> My problem is that my database is big and as the memory consumption<br>> increases as the loop progresses until all my memory is exhauted and the OS
<br>> goes into swap. It seems that the objects from previous iterations are not<br>> been cleared from memory.<br>> Am I doing something wrong, or is it impossible to iterate over a ZODB<br>> database which is bigger than you memory?
<br><br></div>The connection normally only tries to enforce its cache size limit at<br>transaction boundaries: if you are iterating manually in a script, you<br>need to free things up manually, e.g. by calling '_p_deactivate' on the
<br>object when you are done with it.<br><br><br>Tres.<br>- --<br>===================================================================<br>Tres Seaver +1 540-429-0999 <a href="mailto:tseaver@palladion.com">
tseaver@palladion.com</a><br>Palladion Software "Excellence by Design" <a href="http://palladion.com" target="_blank">http://palladion.com</a><br>-----BEGIN PGP SIGNATURE-----<br>Version: GnuPG v1.4.6 (GNU/Linux)
<br>Comment: Using GnuPG with Mozilla - <a href="http://enigmail.mozdev.org" target="_blank">http://enigmail.mozdev.org</a><br><br>iD8DBQFHj3nK+gerLs4ltQ4RAtyIAKDYrXI9Zs8uJhagbudeJG36LdQvaACfe3VU<br>AHgNdjbyIUkRgUkjZsizJMw=
<br>=d42v<br>-----END PGP SIGNATURE-----<br></blockquote></div><br><br clear="all"><br>-- <br>Flávio Codeço Coelho<br>----------------------------------------------------------------<br>"My grandfather once told me that there were two kinds of people: those who do the work and those who take the credit. He told me to try to be in the first group; there was much less competition."
<br>Indira Gandhi<br>====================================<br>registered Linux user # 386432<br>get counted at <a href="http://counter.li.org">http://counter.li.org</a><br>----------------------------------------------------------------