[Zope3-dev] Re: performance and BTrees

[Steve Alexander]

> My idea was to maintain an index as a btree with the thread data using 
> the event service.

You'd want to use the event service.
Write a DateIndex and subscribe it to receive IObjectAddedEvents.
When the object that is added is an IYourMessageThread, index its id 
(its path, I guess) against the date it was created.

If you have lots of other objects being added in your system, you might 
want to send an IMyMessageThreadAddedEvent when your message threads are 
  created. Then, you subscribe your DateIndex to just those events.


> As far as I know a list is sorted very quickly when 
> some unsorted data is append or data is removed. Perhaps you (Marcus) 
> can subclass a btree to make it a self sorting btree.

BTrees maintain their items in the sort order of their keys.


> Is it a good idea to have thousands of objects (here threads) which 
> implements ISubscriber from the event service

No.

> or would it be better to 
> code an adapter which do the caching work and is queried for the data?

I'd have just one central date index for all your threads.
If you also maintain an index of thread->message, you will be able to 
very quickly get the messages for a particular thread sorted by date.

Otherwise, if you want to maintain a BTree with each message thread, you 
can have one subscriber for all your threads that dispatches the event 
to the particular message thread that needs to know about it.


> I suppose the adapter would be better.

I'd be more inclined to make the overall design scalable before 
resorting to caching.

--
Steve Alexander