[Zope] ZCatalog len problem

[Jonathan]

----- Original Message ----- 
From: "Andreas Jung" <lists at zopyx.com>
To: "Jonathan" <dev101 at magma.ca>; <zope at zope.org>
Sent: Saturday, March 24, 2007 10:43 AM
Subject: Re: [Zope] ZCatalog len problem

--On 24. März 2007 10:16:46 -0400 Jonathan <dev101 at magma.ca> wrote:

>> I have a ZCatalog and in the ZMI 'Catalog' tab it reports:
>>
>> Returns:
>>
>> len(sRes)=  1820
>> cnt=  1842
>>
>
> The result remains the same after reindexing the whole catalog?

Yes.  I have an application that has 5 different ZCatalogs. The same problem 
occurs in 3 of them. It seems that a ZCatalog with less than ~500 records is 
ok, but the len problem increases with the number of records in the 
ZCatalog:

actual        len        diff

108           108       0
322           322       0
613           607       6
1842         1820     22
3628         3590     38



>> It looks like the __len__ method in Catalog.py is not returning the
>> correct value - it returns self._length() which is set by the following
>> method (also in Catalog.py):
>>
>>     def migrate__len__(self):
>>        """ migration of old __len__ magic for Zope 2.8 """
>>         if not hasattr(self, '_length'):
>>             n = self.__dict__['__len__']()
>>             del self.__dict__['__len__']
>>             self._length = BTrees.Length.Length(n)
>>
>
> Is this migration code for pre-2.8 catalogs. This change was necessary
> because the new ExtensionClass implementation did not longer work
> with the scary implementation of the __len__ functionality of the Catalog
> class.
>
> If the result remains the same after reindexing the catalog there might be
> an issue with the increment/decrement calls of the _length counter
> with catalogObject/uncatalogObject...however no idea why...

Clearing the catalog and then running 'Find Objects' returns the same 
incorrect number every time (at least the error is consistent!). I have also 
loaded records into the ZCatalogs via a python script/external method and 
the record counts are exactly the same as with the 'Find Objects'.

I don't think it is an indexing problem as the indexes are reporting the 
correct number of entries.  It is just the _length() method of the catalog 
that seems to be the problem (but I have noooo idea how to fix it!)

Should I file a bug report on this?

Jonathan