[ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

[Marius Gedminas]

On Thu, Dec 10, 2009 at 12:41:11PM -0500, Jim Fulton wrote:
> On Thu, Dec 10, 2009 at 11:54 AM, Marius Gedminas <marius at gedmin.as> wrote:
> ...
> > Well, there is IStorage, and I think all storages (except for
> > FileStorage) implemented it faithfully
> 
> I guess that depends on what you mean by faithfully. MappingStorage
> and Demostorage
> in 3.8 inherited a null implementation from BaseStorage and it's
> arguable how faithful that
> implementation is.
> 
> > (oid,
> > version-that-got-removed-in-3.9, size=1).  FileStorage in 3.8 violated
> > the interface by renaming size=1 to length=1, but it's fixed in 3.9.
> 
> and in 3.8.

*checks*

FileStorage in 3.8.1 violated the interface, to be precise.

ZODB 3.8.1 is mandated by
http://download.zope.org/zope3.4/3.4.0/versions.cfg, which our
production app currently uses.

(Kinda offtopic: Is there a newer 3.4 KGS release?
http://download.zope.org/ tells me "Welcome" and nothing more.
http://download.zope.org/zope3.4/ tells me in great big joyful letters
at the top of the page "zope 3.4.0 Has Been Released!")

> > What I miss _a lot_ is a way to look at the last several transactions.
> > There's the iterator API, but it goes from oldest-to-latest, and when
> > you've got a large DB, that takes a while (85 seconds for a Data.fs with
> > 922737 transactions and 8 GB of data, with 100 MB/s read throughput).
> 
> In 3.9 for FileStorage, if you give a starting tid that is toward the
> end of the file,
> it will scan backward, saving a lot of time.

Ah, but how can I get a valid tid that is toward the end of the file?
I assume that if I give a tid that doesn't exist, I'll get an error --
which is the behavior oldstate() exhibits -- rather than the next
existing transaction after that timestamp, or I could fudge by getting
the current timestamp and subtracting 24 hours.

> I've attached an analysis script that illustrates iterating backward.

Thank you!

(Mutt is inconvenient: there's no way to look at attachments while you're
replying to a message.)

> Someday, someone should get around to adding an option to the file-storage
> iterator to go backward.

Marius Gedminas
-- 
Life begins when you can spend your spare time programming instead of
watching television.
                -- Cal Keegan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.zope.org/pipermail/zodb-dev/attachments/20091210/af373b3f/attachment.bin