[ZODB-Dev] Using zodb and blobs

[Nitro]

Am 14.04.2010, 14:45 Uhr, schrieb Hanno Schlichting <hanno at hannosch.eu>:

> Usually you will only loose the last transaction and not a days of
> work. The Data.fs is an append-only file, with one transaction
> appended after another. If there's a garbled or incomplete write,
> you'll typically loose the last transaction. The ZODB is smart enough
> to detect broken transactions and skip them on restart.
>
> I have witnessed one ZEO installation myself, where the physical
> machine hosting the ZEO server restarted multiple times a day, over a
> period of months. Nobody noticed for a long time, as the application
> was accessible all the time and no data had been lost. Obviously this
> wasn't a very write-intense application. But it still showed me how
> stable the ZODB really is.

Yes, I agree with your opinion in general.

There's still a chance that broken transactions are written IIUC (see  
https://mail.zope.org/pipermail/zodb-dev/2004-July/007683.html ):

>> Doesn't this mean that if the system suddenly crashes in the middle of
>> os.fsync, the Data.fs on disk will contain an incomplete transaction,  
>> but
>> the transaction status byte would claim that the transaction is  
>> complete.
>> Wouldn't that be bad?
> If that happened, perhaps.

The chance exists because fsync does not work as advertised on many  
systems. On the systems where it seems to work, the slowdown is massive.  
So I am doubting the usefulness of using fsync in current ZODB at all.

As your observations seem to hint, it's probably very unlikely to  
encounter this problem in practice. And I doubt Tim finally got Jim to pay  
him for a 48 hour pull-the-plug session :-) That's why I am not going to  
dig further into this and am satisfied with the current reliability.

-Matthias