[ZODB-Dev] Re: Fixing Broken Objects?

[Tim Peters]

[Chris Spencer]
[...]
> I'm fully aware of Pickle's limitations. My question was why are you
> accepting them?

Regardless of topic, the answer to that question is usually "although I
don't have time for it in any case, if I did I can think of 100 other things
likely to have better payoff" <0.5 wink>.

> What's to stop someone from extending Pickle to serialize state
> *and* code?

You can do that right now if you're willing to store source code as string
data.  There's much less attraction to storing CPython byte code:

- CPython byte code changes across Python releases:  there's no
  guarantee that a .pyc file created by an older release will run
  under a later release.  In contrast, pickles have always been
  100% transportable from older to newer releases.  Needing to add
  a "oh, unless you pickled a code object" caveat wouldn't be loved.

- You're wishing away all the problems involving imports.  The
  CPython byte code for, e.g.,

    from zope.app.i18n import ZopeMessageFactory as _

  contains the package paths and function names as plain strings.
  How is that supposed to execute if the source code for _those_
  doesn't exist?  There are ways, but they're difficult and
  delicate.

- Very few people _want_ to run without textual source code.
  Tracebacks are much less useful then.  Python debuggers are much
  less useful then.  You can't look at the code to see exactly what
  it does if you have a question.  You can't sanely patch the code
  to worm around a bug.  You can't get on IRC and say "help me
  understand what's going on around line 237".

- Almost all the tools Python users rely on become similarly
  useless, from source control systems to Emacs.

- It's generally thought to be "a feature" that pickle does _not_
  store class implementation code along with that class's instances.
  That's why the Python docs say:

      This is done on purpose, so you can fix bugs in a class or
      add methods to the class and still load objects that were
      created with an earlier version of the class.

  That's very useful.  Of course it creates problems too.  So
  would not being able to "fix bugs ... and still load objects
  that were created with an earlier version ..." easily.


> Python's inspect module certainly allows for this.

Sorry, I didn't understand that (the preceding sentence asked what's to stop
someone from extending pickle, and that's not a reasonable referent for
"this").

> You could define entire programs on the database without the
> client having to maintain any code,

What's the attraction?  Zope Corp runs Zope -- it's not hard to get the Zope
source code ;-)

> only loading what it needs to connect to the database.
> Could it be we're making the incorrect assumption that state
> serialization is enough?

As Jim said, there's some not-fully-baked "persistent module" stuff in
Zope3.  It was difficult to get that to work at all, and apparently nobody
wants it enough to keep working on it.

BTW, what I _really_ want to do is freeze the state of an entire running
application, stuff it on disk, and resume it later.  It never occurred to me
that the reason I can't do that easily is due to pickle limitations ;-).