[ZODB-Dev] [ZODB4] - Shouldn't the new object be added to Connection._created ?

[Ury Marshak]

It took some time to find time to write a trace function ;)

> > Well, unfortunately this is not what happens to me under ZEO.
> > Scenario: a new object is inserted in existing persistent container,
> > container gets pickled, persistent_id()  is called, the new object gets
> > assigned oid. Transaction fails. The new object still has oid and the
works.
>  >
>  > We retry the transaction (or even access any attribute on the object).
>  > The persistent machinery sees the object with _p_state==3, ghost,
>  > tries to unghostify it using a [non-existent] oid -- BOOM..
>
> What works? It can't be referenced from the container any more,
> because the container was ghostified. The new object isn't in the cache,
> so I don't see hos the data could be getting unghostified, or how it could
> be ghostified in the first place. How is the transaction failing?

This is what generates an Exception - trying to unghostify an object that
was not yet properly stored and yet is marked as a ghost.


> >>Note that, if the transaction is aborted, the reference to the new
object
> >>will go away and it will disappear, so it doesn't really need to be
> >>invalidated. You don't need to keep track of created objects until they
> >>have been assigned oids.

My app level code keeps the new object referenced, and then tries to rerun
the transaction after fixing the conflict. As a workaround I could recreate
the
object if necessary, but it might become expensive if the object is already
inserted in some indices


> >>
> > The point is that under ZEO the whole thing seems to take slightly
> > different paths (tpc_abort etc)  than when using for example FileStorage
> > directly.
>
> That is hard to believe.
I wrote a trace function and ran the example using FileStorage and
ClientStorage.
There are some differences (they might not be significant - that's for you
to
say ;-)
Running under ZEO goes like this (I skipped a lot of intermediate function
calls to
cut down the size):
. . . Connection.py:commit()
. . . . Connection.py:commit_store()
. . . . . ClientStorage.py:store()
. . . . . Connection.py:_handle_serial()
. . . . Connection.py:commit_store()
. . . . . ClientStorage.py:store()
. . . _defaultTransaction.py:_subtrans_commit()
. . . Connection.py:tpc_vote()
. . . . ClientStorage.py:tpc_vote()
. . . . . ClientStorage.py:_check_serials()
. . . . . . _defaultTransaction.py:_commit_failed()
. . . . . . . Connection.py:tpc_abort()
. . . . . . . . ClientStorage.py:tpc_abort()

Running under FileStorage:
. . . Connection.py:commit()
. . . . Connection.py:commit_store()
. . . . . FileStorage.py:store()
. . . . . . ConflictResolution.py:tryToResolveConflict()
. . . . . . POSException.py:__init__()
. . . . . . _defaultTransaction.py:_commit_failed()
. . . . . . . Connection.py:abort()
. . . . . . . Connection.py:tpc_abort()
. . . . . . . . BaseStorage.py:tpc_abort()

using FileStorage directly we do not go through Connection.py:tpc_vote(),
but we do call Connection.py:abort() before Connection.py:tpc_abort()


> When you get a conflict error, do you call get_transaction().abort()?
Tried it, didn't really help.. :(
>
> Jim
>

I know you guys are busy, but did any of you heavy-weights have time
to try the sample I've sent before - the relevant portion is only several
lines, all the rest is just the opening the connection, inserting the
initial
objects etc.

Ury