[ZODB-Dev] Advice needed

[Andreas Jung]

--On 24. Juni 2006 08:53:43 +0200 Roché Compaan 
<roche at upfrontsystems.co.za> wrote:

> I am curious what other strategies besides QueueCatalog you employ? Do
> you ever use multiple backends for your apps? How do you decide that
> this data belongs in a relational backend? How structured must the data
> be, or how many records must be written how often?
>
> I find most data is highly structured (fixed schema), but this doesn't
> make me choose an RDMBS - the frequency of writes, concurrency and
> record volume does.

It is often the case where you have more complex data models that really 
require a RDBMS. It is often the case that you have to perform complex 
queries on your data. The ZCatalog is often just too weak and too slow for 
such apps. Our CMS (to which Dieter often refers to) uses the ZODB to store 
large amounts of SGML data. Since the documents fits perfectly inside a 
hierarchy the ZODB is the perfect choice. However we have some apps around 
the CMS that provide additional functionality or use the CMS for a 
different purpose. Some of these apps use the content from the ZODB but 
store their metadata in postgres. One not unimportant advantage is that 
other ppl can run their own reports etc. using a postgres client without 
approaching me to write a script or something in Python to get requested 
data out of Zope.
One particular app that I have been working on uses very complex queries 
with lots of join etc....it would be hard model to implement such queries 
on top of the ZODB/ZCatalog. Another point is performance: this app often 
has to perform a lot of insert/update/delete operations within one 
transaction (up to 1000 modifications). Postgres takes perhaps 5 seconds 
for such a complex operation. You will never reach that performance with 
Zope...touching 1000 objects and reindexing them will take much longer 
(unless you adjust your data modell for performance reasons to the ZODB
needs).

So what I want to say is: the bullheaded idea to stick everything into the 
ZODB or into a RDBMS is just stupid. Smart people think about the data 
storage before starting a project. That also reminds me of some postings on 
the Plone mailinglist where non-technical people put all their stuff inside 
without knowing what's happening under the hood and wonder that everything 
explodes or becomes horribly slow at some point.

Andreas



-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20060624/cdd9fab8/attachment-0001.bin