[ZODB-Dev] Ordering before commit hooks

Wed Aug 31 09:33:24 EDT 2005

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jim Fulton wrote:
> Julien Anguenot wrote:
> 
> I'm gonna respond to a number of emails at once. :)
> 
>> Tim Peters wrote:
>>
>>> [Tim]
>>>
>>>
>>>> ...
>>>> Julien provided links to code that already uses the new feature:
>>>>
>>>> """ As an Indexation Manager :
>>>> http://svn.nuxeo.org/trac/pub/file/CPSCore/trunk/IndexationManager.py
>>>>
>>>> As an Event Manager :
>>>> http://svn.nuxeo.org/trac/pub/file/CPSSubscriptions/trunk/EventManager.py
>>>>
>>>> """
>>>>
>>>> He appears to use (just) two distinct `order` levels there, and seems
>>>> just to want to make sure one class of hook gets run before the other
>>>> class of hook.  The new scheme does give an easy way to do that.
>>>
>>>
>>>
>>> OTOH, while picking levels of -100 and 100 works for that specific
>>> use case,
>>> looks like it threatens to become a mess if multiple subsystems try
>>> to use
>>> this scheme simultaneously.  
>>
>>
>>
>> We discussed quickly about this with Florent. This can be expressed by
>> having a coding guide for the given framework (for instance CPS). We
>> will provide ranges in which hooks are executed. Then if someone wants
>> to register another one he can check the existing ones and the ranges
>> they are using and what they are doing. Just a matter of documentation.
> 
> 
> Of course, this doesn't work across projects.
> 

ok here we agree :)

>>
>>> For example, someone who wants to be "the last
>>> kind of hook invoked" has no way to force that, at least not short of
>>> passing sys.maxint as the `order`.  But if that's what's needed,
>>> `sys.maxint` is a strange way to spell it.
>>
>>
>>
>> I understand it but this is something that could be done by extending
>> the transaction hooks sub-system.  But again we don't have YAGNI yet for
>> a bigger sub-system that could include this.
> 
> 
> The very fact that to really do what you need to do today would require an
> extension indicates that the current API isn't what we should be doing.
> 
> We'll never have what every application needs anyway, which is why this
> ordering logic should be at the application level where it's easy to
> control.
> 
> 
>> I don't understand why keeping the order paremeter is such a big deal ?
> 
> 
> Because it clutters the ZODB API with something that is, fundamentally
> application policy.

Here we disagree.

I could agree if only one hook could be registred instead of a severals
(within a tiny little tuple :) ).
Then you could say : "It's application logic just register your object
as *the* hook that will deal with the order of execution of other
sub-hooks your system defines"

Here, it's not the case because you can register several within a
transaction and actually the ZODB has already a policy regarding this
which is executing it in the order they've been registred.

Thus my proposal could be seen as well as a request to change (or to
make more flexible) the policy already in place. (Which was my use case)

> 
>> It prevents me having to code my own specific object dealing with this
> 
> 
> Dealing with what?  

Dealing with the order of execution.

> Your original need was to make something go last.

Well yes and no. What if I want to go first instead ? Or in the middle
of 2 independently (external to my system) defined hooks ?

> As Tim pointed out, the order api doesn't do that and, as I've pointed out,
> the existing API does deal with that.

no. not enough flexibility.

> 
>> and I'm sure other people will benefit from this later on when they will
>> have to use this transaction feature.

> 
> That's hypothetical. In any case, if you are so sure, provide a system
> that does this and let people use it. You don't need to make ZODB more
> complicated to address this use case.

Jim, honestly where is the complication ?

> 
> Dieter Maurer wrote:
>> Tim Peters wrote at 2005-8-22 16:48 -0400:
>>
>>>...
>>>Jim still wonders, and he got me wondering too, whether the `order=`
> gimmick
>>>is really needed.
>>
>>
>> But, it is a very easy concept -- both easy to grasp as well as easy
>> to implement.
> 
> Just because it's an easy concept, doesn't mean it belongs in
> in the transaction layer.  It would be just as easy a concept
> as a ZODB add-on.  I wouldn't object to such a thing in Zope, for
> example.

Then don't allow the registration of several hooks. Just provide one
slot to register only one. Then I could be fine with this.

Let's imagine :

I got the following hypotetical architecture :

Let's say I follow what you're saying.

- ------
ZODB

Zope3 -> defines hook A (managing perhaps other hooks and manage the
order of execution of its sub-hooks)
z3ecm -> define hook B (idem A)
CPS4 -> define hook C (idem A)
- -------

Imagine within the top layer (CPS4) I want to execute one of my hook in
between two of the Zope3 and z3ecm hooks ? I definitely Can't...

> Julien Anguenot wrote:
>> Jim Fulton wrote:
> ...
>>>Do we have evidence that such applications exist? So far, the
>>>only example I've seen is one where an application wanted a handler
>>>to go last. As I've pointed out in a separate note, this is achievable
>>>without the change.
>>>
>>
>>
>> We have. We are thinking of adding another hook within CPS to maintain
>> our tree caches. (I'm not going into details on the CPS tree cache here)
>>
>> We would have these 3 hooks that we will want to see executed in the
>> following order :
>>
>> IndexationManager -> TreeCacheManager -> EventManager
>>
>> If you want more information about this Jim, feel free to ask.
> 
> Why does the TreeCacheManager need to run after the indexation
> manager? Does it use results from the indexation manager?  if so,
> why can't it call the indexation manager?

In this case maybe. But what if the IndexationManager is at Zope layer ?

> 
> In any case, I don't want to tell you what your application needs.
> But you don't need to modify ZODB to get what it needs.

well... sometimes if it makes sense ;)

> Right, I don't see integers as a good way of ordering things.  I also
> really believe that most applications will be as well served by a facility
> that allows things to go last -- and we can already do that,
> 

go last -> go first -> go in between... whatever.

Cheers,

	J.

- --
Julien Anguenot | Nuxeo R&D (Paris, France)
CPS Platform : http://www.cps-project.org
Zope3 / ECM   : http://www.z3lab.org
mail: anguenot at nuxeo.com; tel: +33 (0) 6 72 57 57 66
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFDFbGjGhoG8MxZ/pIRAs4JAJ0Qt5Pe1MVpQQeXnRj/5inFFfzt8gCeJHoi
LKdflwDPcosJZ6MjErBfE50=
=/UPP
-----END PGP SIGNATURE-----