[Zope3-dev] Florent's O-R blog entry

[Janko Hauser]

Am 23.08.2005 um 20:36 schrieb Shane Hathaway:

> Gary Poster wrote:
>
>> On Aug 23, 2005, at 1:11 PM, Gary Poster wrote:
>>
>> Argh, communication.  That still could be too-easily  
>> misinterpreted,  and I didn't stare at it long enough before I  
>> sent it.  One more try.
>> Meanwhile, deciding that a community project require any specific   
>> backend--Ape, FileStorage, DirectoryStorage, or another--feels  
>> like a  mistake.  Discarding FileStorage or DirectoryStorage, as  
>> Florent  argues, is a significant case of "throwing the baby out  
>> with the bath  water".  We have at least three maintained and  
>> capable ZODB backends,  with different strengths and weaknesses,  
>> appropriate for different  use cases.  Lets not jump to discard  
>> any of them.
>>
>
> I agree 100%.  However, your concern is that projects will require  
> a specific ZODB backend, while my concern is that projects will  
> dump ZODB altogether.  I think the latter is the greater risk, and  
> people need a middle ground so they don't isolate themselves from  
> the rest of the community.  Ape could be a part of that middle ground.
>
> Also, I did not intend to disparage the excellent FileStorage and  
> DirectoryStorage packages.  I always tell people to use FileStorage  
> or DirectoryStorage unless they have a good reason not to, and the  
> biggest reason not to use FileStorage (through-the-web code is hard  
> to put under version control) is already disappearing with Zope 3.

This is a good discussion, and I think this will provide a good  
ground for a technical pro/contra view of the storage situation. But  
I think the post from Florent looks at this from a slightly different  
angle. Perhaps I misinterpret it, but his thoughts look at the needs  
for a content repository storage. I do not think he wanted to totally  
replace ZODB for all the other stuff. And assuming he looks at the  
storage question from this point (actually Florent is in holidays at  
the moment) his views are build with some general concerns as  
background.

Let's assume "enterprise" means big and sellable to corporations,  
then the concerns of potential customers are valid, that valuable  
content is stored in some piece of software, which is only known to a  
small group of developers. Building a content repository as a  
marketable solution on this piece of software needs more convincing  
than to say "We have this piece of great software and your content  
ends in your favorite traditional RDBMS."

Ok I will stop to interpret what Florent may have thought, I better  
present my own path of thinking. In the end I'm against a RDBMS as  
the only core part of a Zope CMS repository.

I started with the general idea to have a content repository for  
simple content objects, which are all described by schemas. This  
leads to a rather flat and more structured, nearly homogenous mass of  
objects, compared to the normal objects present in a Zope CMS.
The repository is a layer over potentially many storages. This leads  
fairly easily to the idea to have a backend storage which stores this  
data into a RDBMS. This is the level Florent probably looked at also.  
But I have concerns to many of the other points. At this level the  
RDBMS is really just a storage of attribute mappings. The hole logic,  
for example the relation between different content objects is part of  
the stored data or held in the repository application or some  
registries. I assume that the moment one starts to use the relational  
aspects of the RDBMS the application logic becomes part of the  
storage. This would  need to be adressed in the O-R-mapper, which  
would mean that also the O-R-mapper becomes part of the application  
logic. There are further proposed benefits of an RDBMS-storage like  
indexing, direct searching, report generation which are all  
reflecting back in the application domain, which would lead in the  
end to the situation that one would circumvent the O-R-mapper for  
complex or special tasks and starts to work directly on the data.  
This in the end is bad from my point of view and greatly raises the  
complexity. It would also mean a big development effort to recreate,  
overshadow and map current functionality given us by Zope  for nearly  
free.

There are many valid points where the ZODB has some shortcomings.  
Blob support for example will be much better, although it will not be  
totally solved by just storing blobs on the filesystem. Which leads  
to my last point. From a "solution" point of view there are many  
hacks or individual adaptions involved to have a big scalable site. I  
think we should look for some of these to be better, means more  
standardly incorporated into the z3ecms toolbox. Just for example,  
the answer to time consuming cataloging for cases with many writes is  
to use the queued catalog product. But integrating it into a system  
is a hand job, needs a developer who knows how to do it, where to  
"fiddle" to integrate it right. Such technically already present  
software needs to be integrated as simple configurable options into a  
z3ecms solution. This needs probably not be done in the Zope3 core  
but in products building atop of this. On the other side these things  
should therefor not be used to argument for a total replacement of  
the storage solution.

As a last note, the concept of a repository as a layer over many  
storages does certainly allow to use an APE-based storage as one part  
of this :-)

with regards,

__Janko