[ZODB-Dev] Excellent overview of ODBMS and my take on ZODB

Tue, 26 Feb 2002 16:59:05 -0600

Great feedback. Thanks. My comments appear below.

---
Patrick K. O'Brien
Orbtech

> -----Original Message-----
> From: zodb-dev-admin@zope.org [mailto:zodb-dev-admin@zope.org]On Behalf
> Of Jim Fulton
> Sent: Tuesday, February 26, 2002 8:44 AM
> To: pobrien@orbtech.com
> Cc: Chris Withers; Pythoncard; ZODB
> Subject: Re: [ZODB-Dev] Excellent overview of ODBMS and my take on ZODB
>
>
> "Patrick K. O'Brien" wrote:
> >
> > Hmmm. Why is that?
>
> Because nobody actually wants it bad enough to to the work to
> make it happen.
>
> ;)
>

I do. And I was planning to do so, since I couldn't find where anyone had
already done so. I'll be starting this in the next day or two, then. Wish me
luck.

> Actually, that's not quite true. Stephan Richter and others are
> working on something called ZOQL:
>
> http://demo.iuveno-net.de/iuveno/Products/ZOQLMethod
>
> I'm not sure how Zope specific it is.

Pretty darn Zopish from the look of it. I couldn't find anything usable
outside of Zope. Oh well.

> > I've looked at the source code and it looked like it
> > would be quite possible to strip out all of the Zope stuff.
>
> Yup.

Good. Then that is what I will do.

> > Or do you mean
> > that the Zope folks have no interest in doing so?
>
> There's plenty of interest, but not enough to make it happen.
> Wriring a query language is no small task. It doesn't help
> that most people's needs are met by either Python or the Catalog. ;)

I'm less concerned about the query language than I am about automatically
maintaining useful indexes. But that may just be a matter of priorities.
Once I solve the indexing problem I'm sure my priorities will shift. ;-)

> > No offense to Zope, but if
> > StandaloneZODB is going to make any headway as an ODB for non-Zope
> > applications it would help to have features like cataloging
> that didn't have
> > a bunch of Zope overhead.
>
> I think that the Zope3 cataloging/object-hub effort will produce
> components that
> will be much more useful outside of Zope than the current catalog.

Hmm. I've been following the discussion and looking at the souce code as you
guys have been checking it in, and it sure looks ... Zopeful ... to me.
Maybe I just don't get it. But then again, shouldn't I be able to understand
it if it to be useful outside of Zope? It sure feels like I have to squint
at the code to be able to see things I could use. They don't exactly jump
out at me the way code in regular python modules do. Object-Hub and the like
seem like they are solving Zope problems, which are not my problems.

My problems are with using StandaloneZODB instead of relational databases.
(And please don't tell me to just use a relational database instead.) I
think StandaloneZODB has tremendous potential and is a heck of a lot more
enjoyable to use than mapping objects to some other storage. I'm having a
blast. I just want more, that's all. But I don't want to have to order Zope
off the menu when all I want is a side of fries, if you know what I mean.

> > I'll also not that the ZODB Roadmap
> > (http://www.zope.org//Wikis/ZODB/ZODBRoadmap) has this to say:
> >
> > "Query and index support
> > Allow automatic indexing of objects, a feature frequently
> requested by ZODB
> > users. Perhaps the concrete form of the project would be to
> integrate the
> > catalog more closely with the transaction mechanism.
> >
> > There is a fair amount of research needed to figure out how to
> attack this
> > problem and to see what previous work we can build on. It might be good,
> > e.g., to build on OQL, which is part of the object data standard, but I
> > don't know enough about it to make an informed decision."
> >
> > I look forward to hearing more about this.
>
> Me too. :)
>
> Building a query language is a lot of work.

I'm sure it is. I've done my share of SQL. I don't really want to try to
invent anything like it at all. I want something much more like regular
Python. Your BTrees already give me a lot of what I need. And Catalog looks
like it has lots of nice stuff as well. For now I'm just going to focus on
removing the Zope overhead. The result might be enough for most cases.

> At IPC8, Aaron Waters presented some work he was doing on Gadfly 2.
> Gadly is a relational database written in Python. It includes
> and SQL intepreter.
>
> Gadfly 2 was going to have some sort of open back end to allow
> alternate storage and indexing systems. This sounded extremely
> interesting to me, but nothing seems to have come of it. I think
> Aaron got busy with other work. It would be great if someone could
> pick this work up.
>
> Another query option is XPath/XQL. These have an edge over SQL/OQL
> because they are good at digging stuff out of arbitrarily deep
> hierarchies.  I don't know if there are well-known indexing strategies to
> support these.

Thanks for all the feedback and suggestions, Jim. I really appreciate it.
I'll let you know how it goes.

> Jim
>
> --
> Jim Fulton           mailto:jim@zope.com       Python Powered!
> CTO                  (888) 344-4332            http://www.python.org
> Zope Corporation     http://www.zope.com       http://www.zope.org
>
> _______________________________________________
> For more information about ZODB, see the ZODB Wiki:
> http://www.zope.org/Wikis/ZODB/
>
> ZODB-Dev mailing list  -  ZODB-Dev@zope.org
> http://lists.zope.org/mailman/listinfo/zodb-dev