[Zope] ZODB and ZCatalog related questions
01 Jul 2002 10:30:41 +0200
On Fri, 2002-06-28 at 22:43, Dieter Maurer wrote:
> Thierry Florac writes:
> > ...
> > 1. I have a new Zope database (= 2.5 Mo). After adding a single PDF file
> > of 3.5 Mo, the database is nearly 10 Mo huge !! Is required space into
> > ZODB "reserved" by big blocks, or may the ZODB's size increase twice as
> > the size of files which are stored into it ??
> Objects are stored as pickles. There are larger (usually slightly larger)
> than the original object.
What are "pickles" ??
What do you mean when you say "slightly larger" ???
> I am astonished that you see your database grow by twice the amount
> one would expect. Try whether packing reduces the size. Maybe,
> the object is accidentally modified.
I've done a really simple test, adding the PDF file as a "File" object
from the ZMI. No property was modified...
But you you mean also that when a single little property like "title" is
modified, it's the whole object which is replicated into the ZODB ???
> > 2. .... Does it means that full text indexing is
> > using as much space as the document itself (but I'm starting with an
> > empty ZCatalog) ??
> Together with each object, the index maintains the set of all its words.
> This is done in order to be able to unindex the object
> without the need to touch every indexed term (but only those contained
> in the object).
> As a consequence, each object takes almost as much space in
> the index as it takes itself (at least for textindexes with many
> different words).
> > 3. I'd like my products instances to be indexed automatically, as soon
> > as they are created or modified. But as indexing seems to be quite long
> > with big files, is it possible to index these documents in the
> > background, so that the user does not have to wait until indexation is
> > finished...
> Shane et. al. has some idea in this direction:
> Perform indexing in a background task.
> Indexing requests are put onto a work queue.
> When Zope is lazy, a background task fetches requests
> from the work queue and executes them.
> You need be a bit careful. Read the ZODB description
> (--> Zope.org) to understand the interaction between
> ZODB connections, transactions and threads.