[Zope3-dev] Queries (was Re: personal conceptual questions)

[Jeffrey P Shell]

On Thursday, May 8, 2003, at 08:07  AM, Shane Hathaway wrote:

> Here's one approach you might take with Ape.  Write a SQL query that =20=

> returns primary keys, then fetch objects by key through an Ape =20
> connection.  It might look something like this:
>
> cursor.execute("select key from shows where "
>                "date >=3D '2003-05-05' and date <=3D '2003-06-06'")
> rows =3D cursor.fetchall()
> objs =3D [jar.loadStub((row[0],)) for row in rows]
>
> There are good things about this (caching, for instance) and bad =20
> things about this (caching, for instance.) :-)

That's good to know.  We were thinking of doing something similar.  I =20=

didn't think of using the jar.loadStub trick.

And again - being able to do this regardless of the underlying storage =20=

would be great.  Some storage types (like APE's file storage or =20
DirectoryStorage) would probably take a significant performance hit, =20
while others could obviously perform great.

>> What would be nice is to be able to have some sort of query language =20=

>> that could go down closer to the storage level and not the =20
>> application level (where catalog is).  This would make working with =20=

>> and querying other storage types easier.
>
> I've been discussing relationships for exactly this purpose.  I want =20=

> relationship (and relational) capabilities in ZODB so that I can mimic =
=20
> them in Ape.  I'd like to hide all SQL from the application.  But I =20=

> just read an article by Fowler that convinced me that hiding SQL is =20=

> not always the best thing to do.  So now I'm considering a =20
> direct-query interface again.  (I wonder if I got the URL from you? =20=

> ;-) )
>
> http://www.martinfowler.com/articles/dblogic.html

You may have.  I've been reading over that, and my current application =20=

design is following similar paths.  The SQL is not abstracted out as =20
nicely as the purist in me would like, but it's enough that the realist =20=

in me is currently satisfied.  I think a difference has been in taking =20=

a component oriented approach rather than looking for a pure =20
object-oriented solution, with components that get loaded up in certain =20=

contexts to be Table Gateways for reading/writing/etc.

One thing that I'm hoping to get out of my current application design =20=

is one that could work well with Zope 3 with relatively few =20
[architectural] changes.  It should work: I've got schemas written in =20=

modules using Formulator that could be converted to Zope 3 Schema =20
objects fairly easily.  I've got edit views acting similar to  =20
zope.app.browser.form.editview.EditForm.  I've got registries that are =20=

stuffed with various Table Gateways which are slated to be responsible =20=

for most of the RDBMS work.  It should all be interesting.  But, like =20=

in the Fowler article, there's a lot of SQL (particularly SQL using =20
advanced query operations) that is more exposed to the app and not =20
hidden in one of the gateways.  I don't think it's a bad thing.  But it =20=

would still allow for more deployment flexibility if a more general =20
Query construct could be used that was independent of how these objects =20=

were stored.

>> I'd be happy with something like this that could query ZODB, and =20
>> could query other storages (via APE or something else) in the same =20=

>> manner.  Not sure if it can happen, but it's another little dream of =20=

>> mine.  Another dream I have no time to do anything about.
>
> +1 ;-)
>
> Thanks for the information about Modeling, though.  I still wonder if =20=

> mixing Modeling and Ape might work.

And Modeling is based on the =FCber-powerful Enterprise Objects =20
Framework, which aims to be a fully powerful storage independent =20
persistence system.  Generally, it's used for O-R mapping, but it can =20=

be used with flat files and LDAP directories as well.  In theory, you =20=

can kick one storage out and put another one in without affecting the =20=

application.  There's a small-business CRM application for Mac OS X =20
that I'm sure uses this.  In single-user mode, its data is kept in a =20
single file.  The same application in group mode uses a relational =20
database.  The application acts the same.  It's pretty cool.  The Fetch =20=

Specification system (basically a Query object, like the pattern in =20
Fowler's P of EAA book) is one of the main powers behind this.

http://developer.apple.com/techpubs/webobjects/Enterprise_Objects/=20
Fetching/chapter_6_section_25.html

http://www.martinfowler.com/eaaCatalog/queryObject.html

The various QueryObject and QueryObjectService proposals I've seen =20
floating around the dev site look like they aim to do something bring =20=

this type of thing to the application level.  I just want to reiterate =20=

how nice it would be if those query objects could (a) be sent to =20
storage level or (b) be 'compilable' for different uses.  (ie - an =20
object 'fetching' service catching a Query, and knowing whether to =20
compile it in to MySQL or Oracle SQL, an LDAP query, or a =20
ZODB/Catalog/Etc query).

*shrug*

It's still early for me after a restless night, and I haven't yet =20
caught up on my Zope3-dev backlog, so I'm sorry if this has been =20
covered before.  Or if I'm not making sense.  Or both.

--
Jeffrey P Shell
jeffrey@cuemedia.com