[ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

[Jim Fulton]

On Thu, Dec 10, 2009 at 11:54 AM, Marius Gedminas <marius at gedmin.as> wrote:
...
> Well, there is IStorage, and I think all storages (except for
> FileStorage) implemented it faithfully

I guess that depends on what you mean by faithfully. MappingStorage
and Demostorage
in 3.8 inherited a null implementation from BaseStorage and it's
arguable how faithful that
implementation is.

> (oid,
> version-that-got-removed-in-3.9, size=1).  FileStorage in 3.8 violated
> the interface by renaming size=1 to length=1, but it's fixed in 3.9.

and in 3.8.

...

> What I miss _a lot_ is a way to look at the last several transactions.
> There's the iterator API, but it goes from oldest-to-latest, and when
> you've got a large DB, that takes a while (85 seconds for a Data.fs with
> 922737 transactions and 8 GB of data, with 100 MB/s read throughput).

In 3.9 for FileStorage, if you give a starting tid that is toward the
end of the file,
it will scan backward, saving a lot of time.

I've attached an analysis script that illustrates iterating backward.
Someday, someone should get around to adding an option to the file-storage
iterator to go backward.

Jim

-- 
Jim Fulton
-------------- next part --------------
A non-text attachment was scrubbed...
Name: transaction_rates.py
Type: application/octet-stream
Size: 2161 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20091210/2ae5afee/attachment.obj