[Zope3-dev] Heads up: bugs in zope.app.catalog?

[Gary Poster]

On Nov 17, 2006, at 7:22 AM, Christian Theune wrote:

> Hi,
>
> Adam Groszer wrote:
>> Hello Christian,
>>
>> Yep, but...
>> What is the `good` behaviour regarding None values?
>> Do we need to catalog them or skip them?
>>
>> Example:
>> If the object is (user.title == None)
>> Shall it be kept in the catalog or not?
>> In case it is not in the catalog I won't be able to search for users
>> with no title. Suggestions?
>> At the moment the AttributeIndex works like this.
>
> Ah, I didn't see that question in your original mail, so you said:
>
>   I found that z.a.catalog, AttributeIndex fails to remove the  
> previous
>   value/object from the index IF the new value is None.
>
> From this I derived that removing the value would be the right  
> thing to
> do. I didn't see the question about whether to index None or not.
>
> IIRC the FieldIndex in Zope 2 uses None as a valid value to index  
> and to
> search for.
>
> If I read the source and tests correctly, it's just an implementation
> accident, because someone used None as a getattr() marker instead of
> using a unique marker like object().
>
> At least there is no test and no interface description that says that
> None should not be indexed.

First, None is classically Python's no value.

Second, as a practical matter, indexes use BTrees to index their  
values, and need both a docid -> value structure *and* a value ->  
docid structure.  BTrees must have homogenous key types to work  
reliably across Python versions.  None has sorted low for a long  
time, I think, so it is easy to forget this, but from both a purity  
and (arguably) a practicality perspective it is a bad idea to rely on  
this.  None is not a string, None is not an integer, etc.  Keep your  
BTree keys homogenous.

Therefore, it is just fine, and even a good idea from the perspective  
of welcoming new Python programmers to the fold, to have None be used  
as the "no value", "unindex me" value.

If you want to be able to ask the question you describe, I suggest  
you explore zc.catalog.  If you use an extent catalog, you can still  
ask the question that Adam poses of the index--see the 'none' query  
type.

Gary