[ZODB-Dev] zeopack error

[Kaweh Kazemi]

Thanks Martijn,

I'll try to dump the pickle, and see what I can find out.

I'm still puzzled how this/what was pickled in the first place.

On 31.01.2012, at 14:39, Martijn Pieters wrote:

> On Tue, Jan 31, 2012 at 14:20, Kaweh Kazemi <kaweh at me.com> wrote:
>> I did the change and here we go:
> 
> Right, that's a ZEO bug report right there then; my change should go
> into ZEO trunk. Jim, did you catch it?
> 
>> I have one assumption - seeing this traceback - but I can't prove if it's correct: one "old" object (which is candidate to be removed) in the users storage is referencing an object in a different storage (we are using multiple databases) which has been packed away previously. Generally that shouldn't happen, but maybe we have wrongly deleted objects from the second storage and packed it previously.
> 
> I think the assumption is incorrect; the method referencesf has the
> following docstring:
> 
>    """Return a list of object ids found in a pickle
> 
>    A list may be passed in, in which case, information is
>    appended to it.
> 
>    Only ordinary internal references are included.
>    Weak and multi-database references are not included.
>    """
> 
> Note that weak and multi-database refs are ignored here.
> 
> A unhashable type error generally means you are trying to use a list
> (a mutable, thus unhashable type) as a key in a dictionary. So, in
> this case it is trying to load a pickle record that shouldn't be
> physically possible in Python; one where a list is used as a key in a
> dict.
> 
> I am a little at a loss on how to continue from here. You could rig
> the .unload() calls in referencesf (in ZODB/serialize) with a try:
> except TypeError handler:
> 
>    u.persistent_load = refs
>    try:
>        u.noload()
>        u.noload()
>    except TypeError:
>        pass
> 
> That'll result in a shorter references list, and thus the risk that
> too many records will be garbage collected. If you are lucky, the type
> error is at the end of the pickle and thus not many references will be
> missed, and/or the missed references point to objects that are about
> to be packed away anyway.
> 
> Alternatively, you could dump the pickle in question (variable p) to a
> file instead of passing:
> 
>    except TypeError:
>        open('/tmp/b0rkenpickle', 'wb').write(p)
>        raise
> 
> Then upload that pickle somewhere for people on this list to analyze.
> I cannot promise anyone will, of course, but if someone could and the
> pickle is shown to only contain primitive data types (no references at
> all) then the pass is certainly going to solve your problem.
> 
> -- 
> Martijn Pieters