[ZODB-Dev] Data.fs size grows non-stop

[Jim Fulton]

On Thu, Dec 10, 2009 at 4:06 AM, Pedro Ferreira
<jose.pedro.ferreira at cern.ch> wrote:
> Jim Fulton wrote:
>>
>> On Wed, Dec 9, 2009 at 10:54 AM, Pedro Ferreira
>> <jose.pedro.ferreira at cern.ch> wrote:
>> ...
>>
>>>
>>> Well, at least we won't have to rewrite the whole bucket... but still, it
>>> would be much nicer to fragment the list
>>> in smaller chunks. We could use an OOBTree instead... but something less
>>> complex would suffice... any suggestions?
>>>
>>
>> It's hard for people to suggest something without knowing your usage
>> pattern.
>> Do you want something that is effectively a set?
>
> The situation is simple: we have events (objects) that have to be indexed by
> date. We currently use an OOBTree with the days as keys and lists of objects
> as values. The objective is to avoid rewriting these lists each time
> something changes, since we can have tens of thousands of events in the same
> day. Some kind of btree-based list or set would work, I guess.
>>
>> Is integer indexing important?
>>
>
> Well, it's an index of dates, so, I guess we could use integers as keys
> instead of strings, as we are now.

I wasn't referring to indexing at that level. You were using lists
rather than sets
and I was wondering whether there was a reason for that.

It sounds like a data structure with minimal container semantics
would be fine to store the events for a day.

If you can arrange that the events have a stable ordering such that
no 2 events are equal, then you can use OOTreeSets rather than
lists.

Jim

-- 
Jim Fulton