[ZODB-Dev] Ordering before commit hooks

[Jim Fulton]

Julien Anguenot wrote:
> Jim Fulton wrote:

...

>>>I don't understand why keeping the order paremeter is such a big deal ?
>>
>>
>>Because it clutters the ZODB API with something that is, fundamentally
>>application policy.
> 
> 
> Here we disagree.
> 
> I could agree if only one hook could be registred instead of a severals
> (within a tiny little tuple :) ).
> Then you could say : "It's application logic just register your object
> as *the* hook that will deal with the order of execution of other
> sub-hooks your system defines"
> 
> Here, it's not the case because you can register several within a
> transaction and actually the ZODB has already a policy regarding this
> which is executing it in the order they've been registred.
> 
> Thus my proposal could be seen as well as a request to change (or to
> make more flexible) the policy already in place. (Which was my use case)

Except that, as Tim and I have pointed out, the order gimmic didn't
deal at all well with the use case of making something go last.

> 
>>>It prevents me having to code my own specific object dealing with this
>>
>>
>>Dealing with what?  
> 
> 
> Dealing with the order of execution.
> 
> 
>>Your original need was to make something go last.
> 
> 
> Well yes and no. What if I want to go first instead ? Or in the middle
> of 2 independently (external to my system) defined hooks ?

What if you want one callback to go immediately after another?  What
if you want a callback to be called only if another wasn't called (or
was called). These are also legitimate policies that the order gimmick
doesn't address.  They are policies that can be handled by a higher-level
plug-in.

> 
>>As Tim pointed out, the order api doesn't do that and, as I've pointed out,
>>the existing API does deal with that.
> 
> 
> no. not enough flexibility.

It's absolutely enough flexibility to make something go last.  It handles
that most convincing use case better than the order gimmick.  It doesn't
handle other hypothetical use cases, and neither does the order gimmick.

>>>and I'm sure other people will benefit from this later on when they will
>>>have to use this transaction feature.
> 
> 
>>That's hypothetical. In any case, if you are so sure, provide a system
>>that does this and let people use it. You don't need to make ZODB more
>>complicated to address this use case.
> 
> 
> Jim, honestly where is the complication ?

The API gets an extra argument who's use is far from clear.  People
reading the API would rightly wonder why they would use it.  What
does it gain them?  If they wanted something to go last (or first),
what argument should they provide?

> 
>>Dieter Maurer wrote:
>>
>>>Tim Peters wrote at 2005-8-22 16:48 -0400:
>>>
>>>
>>>>...
>>>>Jim still wonders, and he got me wondering too, whether the `order=`
>>
>>gimmick
>>
>>>>is really needed.
>>>
>>>
>>>But, it is a very easy concept -- both easy to grasp as well as easy
>>>to implement.
>>
>>Just because it's an easy concept, doesn't mean it belongs in
>>in the transaction layer.  It would be just as easy a concept
>>as a ZODB add-on.  I wouldn't object to such a thing in Zope, for
>>example.
> 
> 
> Then don't allow the registration of several hooks. Just provide one
> slot to register only one. Then I could be fine with this.
 >
> Let's imagine :
> 
> I got the following hypotetical architecture :
> 
> Let's say I follow what you're saying.
> 
> - ------
> ZODB
> 
> Zope3 -> defines hook A (managing perhaps other hooks and manage the
> order of execution of its sub-hooks)
> z3ecm -> define hook B (idem A)
> CPS4 -> define hook C (idem A)
> - -------
> 
> Imagine within the top layer (CPS4) I want to execute one of my hook in
> between two of the Zope3 and z3ecm hooks ? I definitely Can't...

Right. You need to use a common API. So define one and get people to use
it. You *want* the common API to be defined by ZODB.  That's too low level.

> 
>>Julien Anguenot wrote:
>>
>>>Jim Fulton wrote:
>>
>>...
>>
>>>>Do we have evidence that such applications exist? So far, the
>>>>only example I've seen is one where an application wanted a handler
>>>>to go last. As I've pointed out in a separate note, this is achievable
>>>>without the change.
>>>>
>>>
>>>
>>>We have. We are thinking of adding another hook within CPS to maintain
>>>our tree caches. (I'm not going into details on the CPS tree cache here)
>>>
>>>We would have these 3 hooks that we will want to see executed in the
>>>following order :
>>>
>>>IndexationManager -> TreeCacheManager -> EventManager
>>>
>>>If you want more information about this Jim, feel free to ask.
>>
>>Why does the TreeCacheManager need to run after the indexation
>>manager? Does it use results from the indexation manager?  if so,
>>why can't it call the indexation manager?
> 
> 
> In this case maybe.

Maybe what? You didn't answer my question.  I assume you mean that
TreeCacheManager needs results from IndexationManager and could just
invoke it itself.  In which case, we don't need ordering beyond
going last, which order doesn't help.

 > But what if the IndexationManager is at Zope layer ?

What difference does that make?

> 
>>In any case, I don't want to tell you what your application needs.
>>But you don't need to modify ZODB to get what it needs.
> 
> 
> well... sometimes if it makes sense ;)

No, it doesn't.  I'm in favor of the basic hook because it enables
higher level hooks.

BTW, the approach I'm advocating is the approach we took with the
event system.  zope.event defines a *policy free* subscription system.
zope.app.event then builds layers of policy on that.  zope.app.container
provides additional layers.  Applications can (and do) layer additional
policy.

> 
>>Right, I don't see integers as a good way of ordering things.  I also
>>really believe that most applications will be as well served by a facility
>>that allows things to go last -- and we can already do that,
>>
> 
> 
> go last -> go first -> go in between... whatever.

No, not whatever. So far, go last is by far the most convincing *real*
(as opposed to hypothentical) use case and the order gimmick doesn't
work well for it.)

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org