[ZODB-Dev] large C extension objects / MemoryError

[Steve Alexander]

Andrew Dalke wrote:

> Hello,
> 
>   We're working with ZODB outside of Zope.  We are storing
> C extension objects ("compounds") which aren't derived from
> Persistence but do define a pickle.  Meaning they can be
> stored in ZODB.


...but they will not form independently persistent objects.

I can't see that what you're doing is any different than my trying to 
store large python lists. For example, range(insert_large_number).

Although the objects can be pickled, they won't participate in the 
transaction / conflict resolution / transparent retrieval and storage on 
demand systems as independent objects. They will rely on a containing 
object that is Persistent for that.

Your memory error is probably because you're trying to load lots of your 
large objects into memory all at once, because you get them as a single 
persistent lump.

--
Steve Alexander
Software Engineer
Cat-Box limited