[Zope-dev] "zipped" versus "unzipped" eggs (was: Re: SVN: Zope2.buildout/trunk/ Don't pin setuptools.)

[Dieter Maurer]

Tres Seaver wrote at 2008-8-28 15:22 -0400:
> ....
>I don't think zipped eggs are a win in any real scenario, except
>possibly the goofy file-count-limited GAE.  I would strongly prefer that
>you revert this one change (you can override it in your own
>'~/.buildout/default.cfg').

We have observed a drastic speedup (on both Windows and Linux)
in import time (about 30 %) when we have put our complete
(large) application (consisting among others of Zope 2, CMF, Archetypes)
in a (single) zip file.

The probably reason:

  Python's module lookup is very expensive.
  It makes about 5 pokes on each entry in "sys.path".
  These become "stat" for directory entries.

  Looking into an in-memory zip content table is much faster
  than a filesystem "stat".


As the use of eggs drastically increases the number of entries
in "sys.path", Python's imports are likely to get slower.


However, I am not sure whether our observations for a single
large zip (in fact, we use two: one for our application, the other for Python's
runtime library) is valid for the case of many small zipped eggs.



-- 
Dieter