[ZODB-Dev] Spatial indices / zodb standard library

[Nitro]

Am 16.06.2010, 17:55 Uhr, schrieb Jim Fulton <jim at zope.com>:

> On Wed, Jun 16, 2010 at 11:45 AM, Nitro <nitro at dr-code.org> wrote:
> ...
>> Would it make sense to add a default spatial index to ZODB?
>
> No.
>
> Which isn't to say that a spatial wouldn't be a good idea. It just
> doesn't need to be part of ZODB.  While it isn't an urgent priority,
> in the future, I want to make ZODB smaller.
>
> We have a packaging system. We don't need no stink'n batteries
> included. :)

I agree with you about separating ZODB and an indexing package on the  
technical level.

I was more referring to the semantical level. What I miss a bit in the  
zodb/zope/plone world is a focus on certain packages. Usually there are  
1-10 packages for a given task maintained by different people. In most of  
the cases this is a plain waste of resources. Time could often be better  
spent by picking a good default package and then collaborating together to  
improve it.

Imo this also applies to ZODB. During my current zodb related work, I've  
come across a number of useful or interesting packages:

- indexing: zcatalog, zcatalog standalone, IndexedCatalog
- spatial indexing: zgeo.spatialindex, Quadtree
- searching/relations: gocept.objectquery, dm.incrementalsearch ,  
zc.relation, zc.relationship, gocept.reference
- storage history: zc.beforestorage, dm.historical, [zc.vault]
- schema evolution: repoze.evolution, zope.app.generations
- types: zc.blist, PersistentList, OOTreeSet, zc.queue
- there are probably more duplicates...
- many useful packages like zc.zodbgc which have no duplicate

Each time I have to go and read through all the documentation (if there is  
any decent documentation) and then decide if I will use it or not. Often  
the differences are minor. Sometimes I just stumble across the packages  
just by accident. Sometimes stuff is out-of-date. Rather often the  
packaging system breaks, because the .eggs are broken/outdated. Why do I  
have to accidentally discover that there's a zc.zodbgc module to pack  
cross reference databases if zodb core allows me to create cross  
references? What indexing package do I choose? Is that package now for  
zope, plone or zodb?

In my opinion it would make sense to have a ZODB core package and then a  
second package which installs all the commonly used modules. Just like  
python is not very useful without the standard library. I know the modules  
in pythons standard library are good, tested and maintained. Many people  
use them, I can find docs and help.

The zodb standard library should have a default package for  
indexing/searching. One for basic data types like lists, sets and trees.  
One for managing history, ... .

The zodb standard library should then be endorsed by this list/maintainers  
so everybody knows where the focus is. And so I knows who's working on  
which things, so I can possibly collaborate with people. The excessive use  
of packages (on the semantic level) splits a small developer community  
into even smaller pieces and makes collaboration hard to impossible. Imo  
this is also one of the reasons for zodb's weak documentation.

For a start the standard library could be a package which pulls all  
commonly useful, up-to-date 3rd party packages into one place.

Please don't take this as a bashing of any sorts, after all I am still  
using ZODB. I just wanted to show you my thoughts after a few weeks in the  
zodb/plone universe and hope they can be used in a constructive way.

-Matthias