[ZODB-Dev] Memory Errors reading large ZODB

[Jim Carroll]

Hi,  I'm trying to iterate over all the comments in my discussion .fs
and I'm getting different errors each time.  The latest is Memory Error (direct
fs access), sometimes it's trouble starting a thread (access through ZEO.)

I'm running ZODB 3.3 on a FreeBSD system.

I've tried doing periodic commit() and cacheGC(), but the memory use just keeps 
growing until things blow up.

My question is: How can I get my zodb client to forget objects very shortly
after I read them?  (I'm just counting objects, and want to get them out of
memory right away.)

I don't think I can del object because that would also remove it from the
database... isn't that correct?

my main loop is below...

Thanks,
-Jim


    for groupid, group in db.group_db.root.items():

        group_short_name = group.get_user_id()

        # touch each discussion once
        for item in group.get_blog().get_items():
            for comment in item.get_all_comments():
                author_id = comment.author.get_user_id()
                counts[author_id] = counts.get(author_id, 0) + 1

        transaction_commit()
        db.connection.cacheGC()
        print "counted in ", groupid
    #