[Zope-dev] ZCatalog peculiarities?

[Michel Pelletier]

"Phillip J. Eby" wrote:
> 
> I've been spelunking through the ZCatalog source this morning, and there
> are a couple of things I don't understand, that look to me like memory
> leaks and other problems.
> 
> First, in Catalog.py, it seems like virtually every method of the Catalog
> class calls self.useBrains().  useBrains() creates a new class each time
> it's called, and then deliberately creates a circular reference in that
> class, such that it can't be deleted.  Finally, the logic of where
> useBrains() is called, is such that it will be called once for every item
> retrieved from every search done on the catalog.  Thus, every record should
> end up as an instance of a different class, specifically created for that
> record.  This has me baffled, because I can't see why that would be
> something useful.  Am I missing something really deep here, or is it just a
> bug?  It would seem to me that there should be something like
> "_v_brains_inuse =_v_brains" at the end of the useBrains() method, so that
> at the beginning you can check for whether "brains is _v_brains_inuse" and
> skip creating a new class, etc. once the brains are initialized.
> 

The circular reference thing is a hold over from ZTables.  I don't know
why it was there, and I preserved it.  I do think you are right about
the calling logic on useBrains.  I will look into that early this week. 
At the same time, i will experiment with taking out the circular link
and seing if anything breaks.

> The next thing I've noticed, is that Catalog.py imports from Query, but
> then never actually uses anything from it.  Oh, sure, there's an orify()
> function that does stuff, and a query_map local variable set up in
> searchResults() that references orify, but query_map never gets used.  (And
> ZCatalog.py makes up another copy of the same query_map object and passes
> it in, but that never gets used either.)  I even dug down to Index.py and
> TextIndex.py to make sure query_map wasn't used down at that level, and
> then I realized that query_map wasn't part of the namespace being passed
> down there anyway.  This all looks like some sort of vestigial leftovers,
> perhaps during porting from ZTables...?
> 

Probably.  I would have to look at the model, but it might have
something to do with the Z Search Interface.

> The third thing I saw, is that there are lots of constructs of the form:
> 
> if something in somethingelse.keys():
> 
> where the object 'somethingelse' appears to have a perfectly good has_key()
> method available.  Again, is there some sort of deep Zen here, like trying
> to force index pages to be loaded, or...?  I mean, it seems like this would
> be much less efficient than just hitting the has_key() methods.
> 

My Python amateurism showing through there.  I'll make those changes.

> Last, and definitely least, I notice that useBrains() uses a loop to create
> scopy, when "scopy=self.schema.copy()" would suffice.  That wouldn't be
> anything worth 'fixing', except considering how often that code would
> appear to be being called.
> 

This is also code straight out of ZTables.  I will examine your
suggestion.

> I'm really tempted to run something through the debugger or profiler just
> to see if I'm totally off base on how often useBrains() is getting called.
> But I thought I'd run this by the experts, since if y'all did this stuff on
> purpose, you'd be able to tell me.  :)
> 

Well I had some problems with keeping the brains set early on in
development, I may have been overzealous on my calling logic instead of
thinking.  Jim say's thinking is bad. ;)

> Oh, one other thing.  Not a bug, but a feature...  It seems to me that if
> you are searching a ZCatalog on index 'foo', it looks possible to do
> something like:
> 
> Catalog(foo=['bar','baz'],foo_usage='range:min:max')
> 
> And receive records with values of foo ranging from 'bar' to 'baz'.  Is
> this a supported, intended feature, or...?
> 

It is an intended and supported feature yes, but not very well
documented.  This is partuclarly handy for doing, as you deduced, range
searches.  For example, the ZPT has a pull down box on it's advanced
search page that lets you pick date ranges in simple english like "new
since yeserday" and "new since last week" etc.

Note that range searching only works on FieldIndexes.

-Michel

> Just wondering.
> 
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev@zope.org
> http://www.zope.org/mailman/listinfo/zope-dev
> 
> (To receive general Zope announcements, see:
> http://www.zope.org/mailman/listinfo/zope-announce
> 
> For non-developer, user-level issues,
> zope@zope.org, http://www.zope.org/mailman/listinfo/zope )