[Zope3-dev] Florent's O-R blog entry

[Gary Poster]

On Aug 23, 2005, at 3:44 PM, Janko Hauser wrote:

>
> Am 23.08.2005 um 20:36 schrieb Shane Hathaway:
>
>
>> Gary Poster wrote:
>>
>>
>>> On Aug 23, 2005, at 1:11 PM, Gary Poster wrote:
>>>
>>> Argh, communication.  That still could be too-easily  
>>> misinterpreted,  and I didn't stare at it long enough before I  
>>> sent it.  One more try.
>>> Meanwhile, deciding that a community project require any  
>>> specific  backend--Ape, FileStorage, DirectoryStorage, or  
>>> another--feels like a  mistake.  Discarding FileStorage or  
>>> DirectoryStorage, as Florent  argues, is a significant case of  
>>> "throwing the baby out with the bath  water".  We have at least  
>>> three maintained and capable ZODB backends,  with different  
>>> strengths and weaknesses, appropriate for different  use cases.   
>>> Lets not jump to discard any of them.
>>>
>>>
>>
>> I agree 100%.  However, your concern is that projects will require  
>> a specific ZODB backend, while my concern is that projects will  
>> dump ZODB altogether.  I think the latter is the greater risk, and  
>> people need a middle ground so they don't isolate themselves from  
>> the rest of the community.  Ape could be a part of that middle  
>> ground.
>>
>> Also, I did not intend to disparage the excellent FileStorage and  
>> DirectoryStorage packages.  I always tell people to use  
>> FileStorage or DirectoryStorage unless they have a good reason not  
>> to, and the biggest reason not to use FileStorage (through-the-web  
>> code is hard to put under version control) is already disappearing  
>> with Zope 3.
>>
>
> This is a good discussion, and I think this will provide a good  
> ground for a technical pro/contra view of the storage situation.  
> But I think the post from Florent looks at this from a slightly  
> different angle. Perhaps I misinterpret it, but his thoughts look  
> at the needs for a content repository storage.

Hm.   Maybe so.  I'm happy to run with that hypothesis for a while. :-)

> I do not think he wanted to totally replace ZODB for all the other  
> stuff. And assuming he looks at the storage question from this  
> point (actually Florent is in holidays at the moment)

Good to know. :-)

> his views are build with some general concerns as background.
>
> Let's assume "enterprise" means big and sellable to corporations,  
> then the concerns of potential customers are valid, that valuable  
> content is stored in some piece of software, which is only known to  
> a small group of developers. Building a content repository as a  
> marketable solution on this piece of software needs more convincing  
> than to say "We have this piece of great software and your content  
> ends in your favorite traditional RDBMS."

I didn't quite follow this, but from the next paragraph it doesn't  
seem to be the heart of your message anyway.

> Ok I will stop to interpret what Florent may have thought, I better  
> present my own path of thinking. In the end I'm against a RDBMS as  
> the only core part of a Zope CMS repository.
>
> I started with the general idea to have a content repository for  
> simple content objects, which are all described by schemas. This  
> leads to a rather flat and more structured, nearly homogenous mass  
> of objects, compared to the normal objects present in a Zope CMS.
> The repository is a layer over potentially many storages. This  
> leads fairly easily to the idea to have a backend storage which  
> stores this data into a RDBMS. This is the level Florent probably  
> looked at also. But I have concerns to many of the other points. At  
> this level the RDBMS is really just a storage of attribute  
> mappings. The hole logic, for example the relation between  
> different content objects is part of the stored data or held in the  
> repository application or some registries. I assume that the moment  
> one starts to use the relational aspects of the RDBMS the  
> application logic becomes part of the storage. This would  need to  
> be adressed in the O-R-mapper, which would mean that also the O-R- 
> mapper becomes part of the application logic. There are further  
> proposed benefits of an RDBMS-storage like indexing, direct  
> searching, report generation which are all reflecting back in the  
> application domain, which would lead in the end to the situation  
> that one would circumvent the O-R-mapper for complex or special  
> tasks and starts to work directly on the data. This in the end is  
> bad from my point of view and greatly raises the complexity. It  
> would also mean a big development effort to recreate, overshadow  
> and map current functionality given us by Zope  for nearly free.

Yes, you've identified some of my big concerns.  While I am very much  
in favor of O/R mappings such as Ape being a supported back-end for a  
shared community project--and maybe even an optimized one in some  
cases, if someone wants to invest the time to build it--I am worried  
about having a project that begins to rely on it as the primary  
storage, precisely because of the sorts of reasons you mention.

> There are many valid points where the ZODB has some shortcomings.  
> Blob support for example will be much better, although it will not  
> be totally solved by just storing blobs on the filesystem. Which  
> leads to my last point. From a "solution" point of view there are  
> many hacks or individual adaptions involved to have a big scalable  
> site. I think we should look for some of these to be better, means  
> more standardly incorporated into the z3ecms toolbox.

Or the Zope 3 toolbox, or even the ZODB toolbox.  Agreed generally.

> Just for example, the answer to time consuming cataloging for cases  
> with many writes is to use the queued catalog product. But  
> integrating it into a system is a hand job, needs a developer who  
> knows how to do it, where to "fiddle" to integrate it right.

We have an easier to integrate Zope 3 version with a different design  
in the zc sandbox now, by the way.

> Such technically already present software needs to be integrated as  
> simple configurable options into a z3ecms solution. This needs  
> probably not be done in the Zope3 core but in products building  
> atop of this.

I agree that it is the place of projects such as z3ecms to make this  
sort of integration easy (and even sanely preconfigured, if possible).

> On the other side these things should therefor not be used to  
> argument for a total replacement of the storage solution.

Right.

> As a last note, the concept of a repository as a layer over many  
> storages does certainly allow to use an APE-based storage as one  
> part of this :-)

Yes indeed.

Gary