[Zope3-dev] Overriding standard library modules

[Barry A. Warsaw]

I've run into a situation where I need to -- in some cases -- override
a Python standard library module.  I think it /may/ be useful to build
a general mechanism for Zope3 now in anticipation of future needs, but
I'm also willing to hack around my specific need if too many people
cry yagni.

Let me outline three approaches and then tell you which one I'd like
to go with.  First for motivation, the specific problem.

For the Zope i18n stuff we need Python's gettext.py module, but we
really need the version from Python 2.3.  However, Zope3's minimum
requirement is Python 2.2.x, so when we're using Python 2.2, we need
to override the standard gettext.py module with the Python 2.3
version.  We need this to work with Zope both from the build directory
and from the run-in-place directory.

We could do this a couple of ways:

- In current cvs, there's a gettext.py module in src/ and when you're
  running z3.py, you run in place, so Zope picks up this gettext.py
  instead of the standard one.  But this breaks the test suite, which
  by default runs out of the build directory.  src/gettext.py isn't
  copied to build/lib.*.  Also, I think it's fairly ugly to put
  modules right under src/ especially if we end up with more than just
  one or two.

- We could just solve this specific case by copying Python 2.3's
  gettext.py module to zope.i18n and then using

  from zope.i18n.gettext import blah

  rather than

  from gettext import blah

  I don't like this much either because generalized, it becomes more
  difficult to track down override modules when Zope decides to
  support the next version of Python.  IOW, someday Zope will have a
  minimum requirement of Python 2.3, and then we'd like to weed out
  our own special copy of gettext.py and any other override modules,
  otherwise the potential for version skew is too great (see below).

- We could create a directory called pythonlib, stick gettext.py in
  there and then arrange for pythonlib to be on sys.path before the
  standard library for z3.py and for test.py.  This is actually the
  approach I use in Mailman when I have to solve this problem, but it
  has problems of its own.

  For one thing, it's highly magical, a major drawback IMO for Zope
  which has a larger development community.  Second, it can be tricky
  to find all the places where a Python program starts, hacking
  sys.path at the beginning of each.  E.g. what about ZEO's runsvr?

  The big advantage to this approach is that we can just write "from
  gettext import blah" and it'll Just Work.  Then when Python 2.3 is a
  minimum requirement, we can just delete pythonlib/gettext.py and
  it'll Still Just Work.

- Finally, we could create a pythonlib package but not put it on
  sys.path.  Then you'd have to do "from pythonlib.gettext import
  blah" and pythonlib.gettext would do the right thing.  It would use
  a version check to see if the Python you're using has the right
  gettext module and if not, import one from
  pythonlib.compatXY.gettext where X and Y are the major and minor
  revs of the Python executable.  If that raises an ImportError, then
  we just import the global (i.e. standard) module and use it.

This last is what I've implemented and tested, and it seems like the
least gross way of doing things.  It also allows us to drop all
override modules in e.g. pythonlib/compat22 for now, and it gives us
an upgrade path for future Python version.  IOW, when Python 2.3 is
released and we start working on Python 2.4, we may need a
pythonlib/compat23 package.

Thoughts?  Too complex?  Yagni?  I'm pretty sure that we're going to
need something like this, and I'm really sure I don't want something
even more complicated <wink>.

Oh yeah, one other note: this should never be used to provide an
override that differs significantly from a standard module.  IOW,
don't use this for evil. :) The only thing in pythonlib/compatXY
should be copies of modules from the standard library of a Python
version newer than the minimum requirement for Zope.  If we start
diverging Zope's version of standard modules from Python's we'll go
mad at some point down the road when we want to bury the rotting
chickens.  I think the last approach, with the naming scheme, helps
enforce this rule.

-Barry