[ZODB-Dev] Missing loader for multidatabase refs?
jim at zope.com
Fri Mar 30 11:13:47 EDT 2007
On Mar 30, 2007, at 10:10 AM, Sidnei da Silva wrote:
> On 3/30/07, Jim Fulton <jim at zope.com> wrote:
>> > Say there's two databases, and the
>> > object in database '1' is the only one referencing some object in
>> > database '2'. Since those references would be treated like weak
>> > references, wouldn't the object in database '2' go away in a
>> > thus causing a PosKeyError when loading the object from database
>> > that still points to it?
>> Yes. That is a limitation of cross-database references.
> What can be done to avoid that?
By "that" I assume you mean dangling referenced.
You could create extra references in the source database. For
example, if you make a cross-database reference, you could also add a
corresponding reference in the source database.
> Not packing the databases at all?
That will work too. :)
> Would it be possible to make the loading of a cross-database reference
> that is gone return some sort of 'BrokenObject' instead of a
> PosKeyError, to work around the problem temporarily?
Sure. I suppose the same could and perhaps should be done whenever
you would get a POSKeyError.
>> Note that treating the references differently in a pack can't help
>> this as the references are in the referencing database, not the
>> referenced database.
>> It would be cool to have:
>> - A multi-database pack that took multiple databases into account.
>> Such a feature is doable, but obviously, non-trivial.
> Specially if the databases happen to be on different ZEO servers
That is a complication, yes. You would need some sort of distributed
>> - A non-GC pack that got rid of old records but didn't bother with
>> This would be advantagious for lots of folks independent of cross-
>> database reference issues.
> Sounds like this would be the easiest way to solve the above issue?
It is a fairly straightforward way, depending on the application.
Unfortunately, it's probably non-trivial, as the FileStorage packing
code is fairly intense. :)
I've been wanting to redo this code for some time to reduce the
amount of disk I/O. That would certainly be an opportunity to make
GC optional. But maybe it wouldn't be too hard to disable GC in the
current implementation -- I don't know.
>> > Right. I'm wondering how I did end up with a cross-database
>> > Seems like a copy/paste through the ZMI caused it. My question
>> is if
>> > that's expected/correct.
>> Cross database references are pretty transparent and automatic.
>> Maybe there should be an option to make them less so.
> Yes, such an option would be great.
Maybe. It depends on what semantics you want. In fact, I don't know
how Zope 2 is setting up multi-databases. If you don't want cross-
database references at all, you could change the startup code to open
separate databases without making them part of a multi-database.
I suppose there could be could be an option that says you can't have
a cross reference unless the source database has a root key equal to
the object id of the source object with the value being the source
object. Then, an intentional cross reference would be made like this::
# Make a reference to foo, which is in another DB:
foo._p_jar.root()[foo._p_oid] = foo
self.x = foo
The code that creates cross-database references would, if this option
is available, do a check something like:
if other._p_jar.root().get(other._p_oid) is not other:
>> >> What version of ZODB are you using?
>> > The one included with Zope 2.9.6.
>> Hm, are you using ZEO?
>> I'm wondering what sort of release would be needed to help you out.
> Yes, I'm using ZEO. I don't need any sort of release particularly, a
> patch would be fine. As long as there's a Zope 2.9.x release sometime
> in the future for the people that can't live with patches, it should
> be ok.
I suspect you know what the patch is at this point -- at least to get
the pack done. Perhaps you can try it and let us know how it goes.
Jim Fulton mailto:jim at zope.com Python Powered!
CTO (540) 361-1714 http://www.python.org
Zope Corporation http://www.zope.com http://www.zope.org
More information about the ZODB-Dev