[Zope3-Users] zc.tables integration with Zope3
jwashin at vt.edu
Wed Aug 23 09:55:08 EDT 2006
David Pratt wrote:
> Hi Jim. This approach is completely new to me. It is interesting and
> I'll keep an eye on your blog. How expensive is updating indexes with
> data modification. You are talking about sqlite, does this mean you
> figure the indexes would be a significant drag on the ZODB due their
> size or is this idea a performance consideration. Many thanks.
The expense is in recalculating the new factoradic for each stored
index, which involves a lot of division and factorial operations. But
this might be done in a smart way. Maybe instead of on-change, we could
recalculate an index on first access to that index after change.
Sqlite would be mainly for performance, ATM. A lookup is less expensive
than calculating a factoradic of a very large number.
Size may also be an issue if the index is stored in the zodb. An
integer (long as it might be) is the most compact representation of the
factoradic. I'm pretty sure that longs end up as base ten strings in a
pickle, but that is still smaller than the pickle of the factoradic list
representation. I still need to experiment more, but it's not out of
the question that the indexes could end up as sqlite-backed annotations
to the container/data source, particularly when sqlite may already be in
the mix for memoizing factorial and/or factoradic calculations.
It may still turn out that this is all not worth the effort in practice.
More information about the Zope3-users