[Zope3-dev] Re: [Zope3-checkins] SVN: Zope3/trunk/src/zope/schema/interfaces.py - add IIterableChoice and note the desire to depricate vocabularies

[Tim Peters]

[Fred Drake]
> That's not so weird at all; this is how __len__() is generally used in
> Python.  sys.maxint is the recommended value to return if the sequence
> length is undetermined

I haven't heard that recommendation before, and strongly advise
against it.  The result returned by __len__() is often used as a hint
internally, to preallocate result space.  This is exactly what the
"What's New in Python 2.4" doc is talking about in this entry:

    """
    list(), tuple(), map(), filter(), and zip() now run several times
faster with non-
    sequence arguments that supply a __len__() method. (Contributed by
    Raymond Hettinger.)
    """

If you have __len__() methods returning sys.maxint in these contexts
(among others), the best you can hope for is spurious MemoryError
exceptions, as Python's C code tries to malloc() space for sys.maxint
object pointers.  For example,

"""
class C:
    def __len__(self):
        import sys
        return sys.maxint

    def __iter__(self):
        return iter([1, 2])

print filter(None, C())
"""

raises MemoryError during the filter() call.  Have __len__() return
100, or 0-- anything sane --instead, and the correct [1, 2] is
printed.

> (such as when it can only be determined as a side effect of iteration) or
> when it would be prohibitively expensive to compute.  The __len__() should
> normally only be used to provide a hint,

Yes, but a _sane_ hint, please.  sys.maxint isn't a sane guess for how
long a sequence may be.  Python internals generally use (the
arbitrary) 8 in contexts that require a length but where __len__()
doesn't exist.

> unless you know you have an instance of  a type that always returns an
> accurate value (such as a list, tuple, or str).
>
> The Python runtime handles this case; if you find any specific points
> in Python that accept an iterable (which might not provide __len__()
> in the general case) that don't handle not being able to get an
> accurate length, you should file a bug report against Python with a
> test that demonstrates the problem.

Right, but provided __len__() isn't returning an insane value either.