[Zope3-dev] Precompiled configuration: a dead end

[Marius Gedminas]

Zope 3 currently takes about 7.6 seconds to start up on my laptop.  It
is not so bad for a long-running server, but very painful if you want to
run functional tests often.

Of those 7.6 seconds, about 2.1 seconds are spent in __import__ (in
ConfigurationContext.resolve), 3.3 seconds are spent parsing and
processing ZCMLs, and 2 seconds are spent executing ZCML actions.  The
rest (about 0.2 seconds) is spent outside zope.configuration.

I had and idea that it might be possible to shave off a part of that
time by caching preprocessed configuration (more or less the equivalent
of .pyc files for zcmls).

I spent most of last night working on a proof of concept.  The
interesting bit was zope.configuration.xmlconfig.file.  The original
code looks like this:

      include(context, name, package)
      if execute:
          context.execute_actions()

I replaced it with the following (more or less):

      try:
          context.actions = pickle.load(open('zcml.cache'))
      except IOError:
          include(context, name, package)
          pickle.dump(context.actions, open('zcml.cache', 'w'))
      if execute:
          context.execute_actions()

I also had to change a bunch of configuration directive handlers to make
all actions pickleable (i.e. avoid lambdas, closures, dynamically
defined classes, etc.).  I was not entirely successful (unpickling
finally worked, but execute_actions failed after that; I suspect that my
workaround for pickling unregistered global services did not work), but it
was enough to measure the effect on startup time.

By the way, the pickle of all actions (there were 6006 of them) takes
1.4 Mbyte of disk space.

The result of this experiment: unpickling context.actions shaves off
about 0.3 seconds from the startup time as opposed to parsing ZCMLs.
That is, pickle.load eats up 5.1 seconds, while include(...) eats up 5.4
seconds (2.1 for imports + 3.3 for parsing).

It is not worth the trouble.

If someone is interested, I have a diff of my (very very hacky) changes.
It is 1347 lines long and touches 22 files, most of them called
metaconfigure.py.

Marius Gedminas
-- 
Si hoc legere scis nimium eruditionis habes.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zope3-dev/attachments/20040724/57a971e0/attachment.bin