[Zope3-dev] Re: AdapterLookup C optimizations 'timeit' benchmarks

[Philipp von Weitershausen]

Martijn Faassen wrote:
>>>>> By my estimations(i hope i'm not wrong :) C version gives ~20-26%
>>>>> speed
>>>>> up for different methods.
>>>>
>>>> Running the functional tests for a Zope 3 based systems with 434 tests
>>>> yielded (best of 3) a time of 1:12.51 for the trunk and 1:12.32 for the
>>>> branch.  That's about a 0.25 percent improvement.
>>>
>>> That's a bad argument. Several projects only use zope.interface and
>>> its adapter registry (for example twisted). For them a 20-26%
>>> improvement for the adapter lookup would be significant. Just because
>>> other parts of Zope 3 are even slower and use more time than the
>>> adapter lookup, it does not have to mean that this is a
>>> bad/insufficient improvement.
>>
>> The tradeoff is complexity / loss of maintainability vs. speedup.  If
>> Benji's measurements hold up, we would be taking on the burden of
>> maintaining a parallel C implementation (in sync with the "canonical" C
>> version) within our project, for the (possible) benefit to other
>> projects.
> 
> I also hope that in this case (unlike some other cases in the Zope 3
> source code), a parallel Python version *will* be maintained and
> continue to be tested. I ran into some problem with non-functioning
> stuff (debugging logs) in the security implementation, most likely
> because of a C implementation getting hooked in after the original code
> was written (compounded by there being no tests to verify whether the
> security logging facility worked).

I think we should see to it that in cases of parallel implementations,
unit tests are set up to run twice, once for the Python implementation
and once for the C one. That way, by the rule of checking in no code
that produces unit test failures,  you are forced to modify both
implementations when you modify, fix a bug, etc. in either one.

> The big question is of course what the impact on the performance of a
> typical Zope 3 application will be. If it's minimal it's not really
> worth it.
> 
> Note that projects like PyPy are getting there slowly but steadily --
> it's possible that in a year's time we may only need to do minimal
> tweaking of Python code to benefit from compilation-level benefits. In
> the mean time, Pyrex is another avenue that might be worthwhile exploring.

Pyrex is sexy because it only requires the basic knowledge of how C
works (types and data structures) and lets you code in Python syntax.
That is especially useful for projects like ours where not everyone is
sufficiently familiar with C but could still spot and potentially fix a
bug in Pyrex code.

In conclusion, I think exploring optimizations is good, as long as we
have a Python "reference" implementation that I can understand :). In
the end, it comes down to what Tres said: does the speed improvement
outweigh the extra complexity?

Philipp