[ZODB-Dev] Loading objects directly by OID

[Shane Hathaway]

Greg Ward wrote:

> On 05 November 2001, Barry A. Warsaw said:
> 
>>>From a practical standpoint, I think the answer is yes, this is
>>possible.  I say this because I've seen this happen in my
>>FS->BDB migration tests.  I take a FS for real live data, use
>>copyTransactionsFrom() to create a BDB from this, and then do a pack()
>>on the BDB database.  I've definitely seen pickles containing
>>references to other objects (oid) that are not in the database.
>>
> 
> That's interesting to hear.
> 
> I don't think that's *exactly* the situation I'm encountering, though.
> Note that I worded my question quite carefully:
> 
>   ... but for B to not be loadable via the storage's load()
>   method?
> 
> B (the object referenced by object A), certainly appears to be in the
> database.  It gets loaded if I follow object references to get to it,
> eg. if I do dir(A).  But I can't load it by passing its OID to the
> storage's load() method.


This sounds intriguing.  I thought I'd add my 1.5 cents. :-)


> I'm going to try to strip the database in question down more -- I
> already got it from 80 MB down to 2 MB, but there are still a couple
> thousand objects in it.  Then maybe I can throw it at one of you guys
> and you can tell me if our storage is (gulp) corrupt, or if I'm just
> making incorrect assumptions about the nature of FileStorage.


Note that "corrupt" has many meanings when it comes to FileStorage, and 
in fact it's pretty difficult to actually lose data with FileStorage. :-)

- If a class is missing or a class that used to derive from Persistent 
is no longer Persistent, ZODB will choke but it won't stop your 
application.  (You should always enable logging so you can see this 
happening.)  You can usually recover just by putting an old module back, 
at least temporarily, and restarting.

- Unless there are bugs, during normal operations there is no way to get 
a dangling reference.  But even if you get a dangling reference, you can 
recover from it just by loading the parent object, being careful not to 
touch the bad child, deleting the reference to the child, and storing it.

- If the computer loses power while writing a transaction, FileStorage 
will not use the partly-written transaction.

- If you write a transaction that does major damage to the application, 
you can just truncate that transaction.

FileStorage may not deal well with data stream errors, but if that's an 
issue you can use RAID to get redundancy.

Hmm, it might be useful to have a little utility for verifying the 
integrity of a ZODB.  It would just make sure there are no dangling 
references and no bad class references.  Would it help you?

Shane