[Zope3-dev] personal conceptual questions

[Marcus J. Ertl]

Hi, it's me again!

I'm very afraid of being someway interfering with my personal problems!
But this is the only place I can realy talk about Zope3 and ask my
sometimes stupid questions.

My not at all new problem is more conceptual then realy a concret zope3
problem.

What I try to write is a threaded board. After some versions, which use
postgres as backend for stroring all messsages, I decided after talking
to some of yu, to give a storing off all the data in Zope3 a try. This
solved many of my problems of integrating this data without loosing
Zope3-functionalities! All things worked fine.

I have now following objects:

* The Forum, just a folder, with some views, holding the boards!

* The Boards, folders, holding the messages, only few is special about
them!

* The Messages: They are folders to, containing other messages,
building in this way the structure of the threads with its parentship
of replies!

This was very easy to do, and I'm getting a powerful administrativ
interface for free... Zope3 is great!

But then I put 2000 threads, containing all together about 20.000
messages into my board. And at this point some thing got terrible slow!

If I build the list of treads, I have to sort 2000 threads by date, and
that got very slow.

No problem, after discussing it in this mailinglist I decided to build
a index by date, using the messageservice to keep it in sync.

This should work. I just can take out the slice I need for showiing the
acual requestet browserpage. But than I thougt a bit about it:

I'm starting to use workflow for approval of messages by moderators, so
I have filter out the unapproved messages. But at this time I hade to
filter all 2000 threads b their state... not that bad, but someway
timeconsuming. No problem, I may build a second index by date, only
keeping "user visible messages".

Next step: Moderators should get a list of unapproves messages, for
which they have the right to moderate! That's a realiy bad thing,
because I had to walk trought all the 20.000 messages, getting their
state.... have I now to build a third index of unapproved messages?

Next thing: On users page, each user should have a list of *all* of his
messages and their state. So, I have to recurse througth all messages
in all boards! Hmmm, sounds like I need a index of messages by owner,
that's the fourth index!

Have I realy to build a index for each of this things? And have to keep
it in sync by my own? Isn't there a easy way to get to my messages?

I think, we realy need something like SQL for Zope3, calling it ZQL...
a Zope Query Language, or a Query Service (I read some pages about it,
but got no feeling how it should work!).

I need an easy way to ask for objects recursivly under a certain
subfolder, using all of the objects direct and indirect (accassible per
adapter) properties.

something like:

  list = ZopeQuery(self, "owner='myself' and workflow.state='pending'
sort by date limit 20 offset 40")

Yes, looks like SQL, because I know SQL... LDAPlike syntax or something
like XQuery would be fine too, but then I had to learn one more thing.
But it should be performant, very perfomant!

Or would it be better to store my data in postgresql, using the hubid
of the ObjectHub as ID, and using the ObjectHub for putting together
zopeobject for building the hierarchy of messages, and data in the
database? In this case, I could use Zope Objects for handling the data,
but could take fast sql-lookups for retrieving the set of messages from
my database. But in this case, I see problems arriving from properties
not stored in postgres, like the workflow state.... It would be a
horrible between to worlds...

Sorry about that long message, but I'm fighting with this problem, and
can't get to a decision, which way I hade to go!

Bye and thanks
   Marcus
-- 
 Das LARP-Portal im Internet:
LARP-Welt: http://larp-welt.de

!fantasy