[ZODB-Dev] High Write Applications

[Chris Withers]

Phillip J. Eby wrote:
> The root causes is that RDBMS transaction logs record high-level 
> details, not blob snapshots.  For example, when an RDBMS logs that you 
> changed row #2967's "foo" column to value "bar", it doesn't usually also 
> log the entire data page the row was contained in, plus copies of all 
> the b-tree index pages that changed as a consequence of the change.  
> Thus, ZODB's disk usage per write transaction generally exceeds RDBMS 
> disk usage for the same transaction by at *least* an order of magnitude, 
> even before catalog indexes come into play.

oh :-(

> To do this, ZODB would have to be able to understand and log 
> *differences*, rather than just snapshotting object states. 

How would it go about doing this?

> And, it 
> would need to be able to manage periodic checkpointing, so that 
> recovering a database wouldn't require rerunning all the transactions 
> that had ever been done on it.

Hmmm, I'm no expert in this kind of thing. How does periodic checkpointing work?

> By the way, if you've looked at systems like Prevayler,

I haven't ;-) Where can I read more?

cheers,

Chris