[ZODB-Dev] ZODB Ever-Increasing Memory Usage (even with cache-size-bytes)

[Alan Runyan]

On Tue, May 18, 2010 at 1:14 PM, Ryan Noon <rmnoon at gmail.com> wrote:
> Hi All,
> I converted my code to use LOBTrees holding LLTreeSets and it sticks to the
> memory bounds and performs admirably throughout the whole process.
>  Unfortunately opening the database afterwards seems to be really really
> slow.  Here's what I'm doing:
> from ZODB.FileStorage import FileStorage
> from ZODB.DB import DB
> storage = FileStorage('attempt3_wordid_to_docset',pack_keep_old=False)
> I think the file in question is about 7 GB in size.  It's using 100 percent
> of a core and I've never seen it get past the FileStorage object creation.
>  Is there something I'm doing wrong when I initially fill this storage that
> makes it so hard to index, or is there something wrong with the way I'm
> creating the new FileStorage?

Is there a 'index' file that is being created?  It would be in the
same directory as the database file.

How are you closing the application?

If you see the index file changing when you start up; it is probably
rebuilding the index.

-alan