[Zope3-dev] Context and Component Lookup

[Phillip J. Eby]

At 05:23 PM 8/6/03 -0400, Jim Fulton wrote:
>Phillip J. Eby wrote:
>
> > just as a component that is "part of" a radio would expect to
> > receive its "power" utility from the radio it is composed of.
>
>If I borrow my neighbor's DVD movie,
>I'll play it with my DVD player and radio, even though it's still
>their DVD. :)

But you have now changed its context, in a way that I'd consider equivalent 
to moving a content object.  But, we were talking about *using* 
something.  To modify your analogy, I understand this to be more like you 
going to your neighbor's house to watch the DVD on his player.  While you 
are there, you should respect the rules of *his* household - i.e., context.


>>Maybe I'm misunderstanding what you mean by "use".  I can think of two 
>>kinds of use:
>>1) Present a foreign component, using local presentation rules
>>2) Invoke non-presentation operations upon a foreign component
>>#1 obviously requires that I control where presentation adaptation is 
>>done, but that's okay because the foreign component doesn't need to know 
>>anything about that.  For #2, it seems downright reckless for me to try 
>>to do some kind of hidden parameterization by changing the foreign 
>>component's operating environment while I'm calling it.
>
>Maybe the difference is that my components are connected soley
>through interfaces. Your connections tend to be more intimate.
>
>Generally, the dependencies are on interfaces that must be provided in
>either case. So I don't see anything "reckless" here.

And that makes it clearer that we aren't quite on the same page.  My point 
was that if the "foreign" component does a utility lookup on itself, and 
gets a different result based on who called it, that this isn't a good 
idea.  For example, this forbids the "foreign" component from caching the 
result of such a lookup between calls.


> > If I need to
>>parameterize the component, why not just make an instance of it that's 
>>attached to my environment, or else explicitly define parameters as part 
>>of the call signature?
>
>We're talking about sharing content components.  So we get a bit of content
>from S2 that we want to use in S1.  We may need to adapt it to something else
>we need. I see no harm in using our adapters.

Me either!  What I am saying, though, is that if there's code in S1 that 
does this, why can't the S1 code do so explicitly?  Here, S1 is the 
consumer of the adapter, so it makes perfect sense for it to use S1 as the 
lookup context.  What I don't want is for this to have any impact on *the 
S2 code's execution*, which is what I understood you to be proposing.


> > It seems like an invitation to debugging hell to
>>have things controlled by some kind of threaded global.
>
>And I've experienced debugging hell because objects didn't have the
>context I expected them to have.  With what I propose, the source
>of components is much easier to predict that the way things are now.

And if the context is explicitly specified, it'll be even easier to 
predict, right?  :)


>>But do you have any other use cases, besides presentation?  I don't.
>>But maybe there is something we don't agree on regarding what it means to 
>>"use" a component?
>
>Well, I might have a component for converting between formats.  The two sites
>might have different components.  I want to use the converter for the site
>I'm using the object in.

That's cool, as long as it's the *consumer* of the foreign component that's 
doing the lookup, and saying where to look it up from.


>   Also, I'm moving in the direction of unifying many
>of the component types.  For example, a view is just a named adapter from
>an object and a request to some Often empty) interface.
>
>I can think of a pretty comelling example that supports your point of view,
>however. If I modify an object from S2 in S1, I probably want the events to go
>to S2.  Then again, it's likely that S1 and S2 will share a common event 
>service,
>so it won't matter.

That's the kind of thing I'm talking about, yes.  Interference with an 
object's lookup context is "invasion of privacy" aka breaking 
encapsulation.  The object's author knows more than you do about what his 
object needs/wants.

I am saying there should be "freedom of choice" (objects that do lookups 
can choose where they want to look up from) but no "invasion of privacy" 
(other objects should not have the ability to reroute an object's lookups, 
unless they are the object's owner or context, or have been explicitly 
delegated this responsibility in some other way).



>>>Sure. That's just an example. The point is that there are many components
>>>that don't have containment information, many of which don't really have
>>>a place.
>>
>>But why do they need one?
>
>We need a basic for looking up components.  If the component's place is the
>basis for looking up other components and the component doesn't have a 
>place, then
>that's a problem.  OTOH, if we always look up components based on the place we
>initiate the computation, we have an unambiguous place to do the lookup.

Okay, let me see if I understand.  What you want, is that code that calls 
Z3CA functions like getAdapter, getUtility, etc. should not have to know 
what "context" to use.  Or, in other words, objects should be able to use 
the Borg's communication system without needing implants.  :)



>>>No, because the component lookup may be far removed from the
>>>presentation code.
>>
>>*confused look*.  I think I really need a real (i.e. motivating) use case 
>>to understand this.
>
>You have some presentation code that calls an operation on a content 
>component.
>This operation calls other operations, possibly on subobjects or objects
>returned from other operations.

I don't see any motivation here.  Why do I (presentation code) want to 
interfere in those objects' private business?  If they're not doing 
presentation operations, what business is it of mine?  Presumably, the 
objects are separated into presentation and non-presentation code for a 
good reason.  If the non-presentation objects need to call into 
presentation code, then there should be an explicit parameter passed, such 
as a subcriber for notifications.

What I mean by motivating use case, is a specific thing that some piece of 
Zope 3 code is currently doing, that is ugly because of the way things work 
now, and would be made better by having this implicit contextness -- and 
wouldn't be just as much better if objects could conveniently manage their 
own context, e.g. by an attribute pointing to their container.


>This is getting circular.  I don't want an object to need to be context
>aware to find components. A component should be context aware if *it*
>uses context itself in some way, as sometimes happens.  If component lookup
>requires context, then any content component that might lookup components
>must be context aware.  That's why I don't want component lookup to require
>context.

What I don't get, is why content components shouldn't be allowed to do a 
context-free lookup in that case.  If the object doesn't have any context, 
then presumably its author *wanted* it that way.  No implant, no Borg 
hyperspace communications.  ;)  Heck, it's not like there can't be an 
interface to implement, such that no DNA modification is required, just an 
external adapter.  :)



>>>  but I'm not comfortable
>>>requiring special DNA to work with components.
>>
>>...and then I don't get this.  What are you saying is the "special DNA" 
>>to "work with components"?
>
>If component lookup requires context and if a component wants to do component
>lookup, then it must be context aware. It must have the context awareness DNA.
>I don't want to require this DNA and, thus, I don't want to require 
>context for
>component lookup.

So, if you don't have context, use only global services.  Aren't most Z3 
services globally registered anyway?



>>(For that matter, I don't see why you think *proxies* are necessary for 
>>objects to have context.  Adapters are quite sufficient; for example in 
>>PEAK I have an adapter for ModuleType so that modules can be considered 
>>components which have their package as their context.)
>
>Adapters *are* proxies.  The proxies I'm refering to are also persistent,
>which is usually not the case for adapters.

By proxy I meant "thing you pass around everywhere *in place of* the 
original object, that is expected to conform to any interface the 
underlying object supports".  I.e. Acquisition wrappers, context wrappers, 
security proxies, the new decorators, etc.  An adapter, by contrast, is not 
expected to supply all the other interfaces of the object it adapts.


>>>I agree, but should context awareness be a prerequisite for working with
>>>the component model? I suggest not.
>>
>>I guess I'm confused on why you think this is required.
>
>If component lookup requires context, then context awareness is required to
>do component lookup.

This is the bit I don't get...  If somebody is *writing code to use the 
Z3CA*, and they're writing a *persistent object*, they've already shown 
themselves quite willing to join the Borg.  They're not going to object to 
a painless little implant.  :)  I think maybe you're assuming that because 
ContextAware is a PITA to use *now*, that this would still be true in the 
case of containergeddon or something like it, where objects just keep track 
of their "one true parent".

For example, suppose that being ContextAware meant nothing more than that 
you have to have a __parent__ attribute?


>>>That works great to the extent that we get to objects via traversal, but
>>>traversal isn't the only way to get to objects. We often get to objects
>>>via method calls. We have been forced in many cases to make the methods
>>>wrap their results.  This is a lot of bother.  It significantly slows
>>>development.
>>
>>*lightbulb goes on, briefly*  I think *maybe* I understand what you're 
>>saying, but let me see if I can paraphrase it:
>>1. You need to look up components in the context of some other component, 
>>either utilities, adapters, or "other".
>
>I'm not sure what you just said.  I have components, typically content
>components, for which I may need to look up adapters, and other components.
>I can't control whether these components for which I need other components
>have context.
>
>
>
>>2. Because components cannot be generally assumed to be context aware, 
>>*all* code ends up assuming a burden of supplying context to the objects 
>>that are being passed around.
>
>Yes, that's a way to look at it.
>
>
>>I think I've managed to avoid this problem in two ways...  first, I 
>>almost never look up utilities in the context of another object than self.
>
>That doesn't tell me anything. What's self? What't to assure that
>self has a context?

This is what clued me in to the fact that you're trying to save the Borg 
joinees from having to have implants.  :)  IMO, if I had to subclass 
Persistent and I'm importing stuff from zope.component, I've already joined 
the Collective.  Hook me up a __parent__ attribute or whatever so I can 
assimilate, already.  :)


> > Instead, the responsibility for providing the utility is placed
>>on the interface of the object that would be the context for such a 
>>lookup.  That is, utilities should be part of an explicit interface, or 
>>the use of them hidden within their client.  (The supplying component may 
>>then look up the utility in its own context, but the point is that it's 
>>the supplier's responsibility.)
>
>I'm sorry, I've read this about 5 times and it doesn't make any sense
>to me.

Let's say I come over to your house, and would like to consume some food 
from your refridgerator (access a utility).  It should either be part of an 
explicit agreement that I can take things from your refridgerator (and if 
so, what I can expect to find there), or we should agree on what food you 
will supply me with when I come over, and then you go and get it from the 
fridge for me.  :)

In other words, it's a little forward for objects to look things up in 
other objects' contexts.

(OTOH, I see the more worrisome issue of thread-based context as being like 
all your friends re-organizing your refridgerator whenever they come over, 
such that you can't find a darn thing whenever they're around.  You could 
reach in for a beer and end up drinking photographic development fluid, or 
something.)


>>Second, for adaptation, the caller is always in effect required to know 
>>the context for adaptation, because it is the caller's context that 
>>desires the functionality.
>
>You are assuming that the caller is aware of context in the first place.
>I want to avoid requiring that.

This is now clear to me.  :)


>>Now, it may be that these approaches can't work for Zope 3 for some 
>>reason; maybe there is a lot more need for objects to poke into each 
>>others' business that I'm not understanding.  If somebody can point me at 
>>some code that they don't believe this approach works for, it would help 
>>my understanding tremendously.
>>Specifically, a counterexample would be a situation where either:
>>1. An object A looks up a utility in context of object B, but it does not 
>>make sense to explicitly require IofB to provide the utility as an 
>>attribute or via a method, or
>
>No, an object need to look up a utility. For it's own use.
>Now, it calls getUtility(self, IFoo).
>
>Now suppose that the object isn't context aware. If that's the case,
>it will get a global utility.
>
>Finally, assume that all of this happened in a method of an object that
>was being used in some site that has an applicable utility, but the site's
>utility isn't used.  This is liklely to be surprising to someone.
>
>Perhaps there isn't a global utility, in which case the call fails even though
>there is a matching utility in the callers's site.

Understood; this is where we disagree, because I believe components doing 
lookups should know their own context, and that this is the natural price 
of being able to do a context-driven lookup.  Maybe it should be a cheaper 
price than it is currently in Zope 3; I know that my experience of doing so 
in PEAK is that it's a price more than worth paying, and a price that can 
also be lessened with adapters.  OTOH, I *don't* work with persistent 
application-level components (as opposed to application *data* components), 
so maybe there's a big difference there.  But I find it hard to see what 
the big deal with having a required attribute is, anyway, given that 
there's already such things as __implements__ and __provides__.


>>2. An object A gets an adapter (or view) of an object B, and it does not 
>>make sense for A to use itself, or some known context-aware collaborator, 
>>as the context for the adapter/view lookup.
>
>Well, lets consider some possibilities.
>
>Let's suppose A calls getAdapter(B, IFoo, context=A).
>Imagine A is from S1 and B is from S2.  Now we'll adapt
>B using an adapter from S1, which won't make you happy.

On the contrary, that's exactly what I think it *should* do, for an adapter 
lookup, although I'm not averse to it using B as the context, 
either.  Using B as context for a utility lookup, though, is a bit like 
being a pushy guest, grabbing a beer out of B's fridge without an 
invitation.  :)  If I am to be allowed to do that, it should be part of the 
documented interface (invitation).


>Or, perhaps A doesn't have context information, in which case
>we'll either get the global adapter, or we'll fail. At that point,
>probably no one's happy.

Ah, but it's not anybody else's business to make A work.  If A wants 
context, A should know *what* context A wants, or else we're just doing 
guesswork.  "In the face of ambiguity, refuse the temptation to guess".


>Now suppose instead, A calls getAdapter(B, IFoo). If B has no
>context, then either we'll get a global adapter or fail. Again,
>no one happy.

But then, why isn't it defined as a *requirement* that B have a context in 
that case?  Why can't I say, "hey, if you don't have beer in your fridge, 
I'm not coming over"?  :)  In my experience, though, it's pretty rare for a 
client to want to use a supplier's context for an adapter.  Presumably, I 
am adapting to meet *my* requirements, so my context would "know best" what 
adapter is suitable, no?

As I said, perhaps my experience doesn't mesh with what Zope 3 code 
actually does - and I'll be the first to admit my knowledge of day-to-day 
zope.app coding is very limited.  But I would like to understand *why* you 
would want to use the thing you're adapting, as the context for the 
adapter, and yet *not* make that part of your interface requirements for 
the object you're getting it from.



>If B does have context, we'll get the adapter from B's context.
>At that point, you'll be happy, but I probably won't be.

I doubt that I'll be happy, unless I had some special reason for wanting 
B's context to supply the adapter, and at the moment I can't think of any 
reason for wanting that.



>>In my understanding, it seems that #2 would almost always be resolvable 
>>to either a) it's a presentation-related adaptation/view, and belongs in 
>>a context based on the request,
>
>Do you mean that A is presentation code?

Yes.


>But what happens if A calls non presentation code? Should the rules
>change?

No.  If I want that non-presentation code to do something presentation-ish, 
I should supply explicit parameters.  If I want to adapt the 
non-presentation object in some way, I should use my context for the 
adapters, because I'm part of a context that has some intended way of 
presenting things.  If I use the non-presentation object's context to find 
presentation components, I could end up using the wrong presentation.

Now, if I need the non-presentation component to point me to some other 
non-presentation component, then of course I need its context, *or* make 
the information I need an explicit part of the interface between the 
presentation and non-presentation component.


>  or b) it's not presentation-related, and
>>object A should use itself (not object B) as the context for adaptation.
>
>So, suppose that we have a view in site S1. Any object it accesses
>should be adapted by adapters from site S1.

Agreed.


>If the view accesses an operation on a context aware object from S1 and that
>operation adapts an object from S2, then an adapter from S1 should be used.

Yes.


>If the view accesses an operation on a context aware object from S2 and that
>operation adapts an object from S2, then an adapter from S2 should be used.

Yep.


>If the view accesses an operation on a context aware object from S2 and that
>operation adapts an object from S1, then an adapter from S2 should be used.

Sounds good so far.


>If the view accesses an operation on a non context aware object and that
>operation adapts an object from S1 or S2, then a global adapter should be 
>used.

Right on the money.


>This all seems pretty arbitrary and likely to lead to debugging hell.

This seems to me like, "the object that wants something should decide where 
it wants to look it up from, and ordinarily an object wants to look things 
up with itself as the context".  Looks perfectly consistent across the board.


>>>Worse, developers usually don't understand why it's necessary.
>>>Something doesn't work, so they look into their book of incantations.
>>>The
>>>book suggests that maybe a context wrapper will work, so they offer one up.
>>>To them, it's a dead chicken.
>>
>>I think that will be helped a lot by making it explicit, and *not* a 
>>"transparent" wrapper.  If an object is expected to supply a context for 
>>something,
>
>Why must we expect an object to supply a context for something?
>Generally, the thing supplying context is traversal.
>We get objects lots of other ways. When a method returns a value, most of
>the time, all it cares about providing is the value itself. It doesn't
>*care* what you do with it.  Some methods provide context for their return
>values. Where this is done, however, it's to supply context for
>context-aware objects that use context for something else.

I think I'm better understanding what's at issue...  Z3CA lookups, apart 
from utilities, default to using the subject of adaptation as the 
context.  Whereas, for anything other than utilities, I prefer to use an 
explicit context (that rarely has any connection to do the subject of 
adaptation).

Because of this, I don't run into the issue you describe, because there is 
almost never any reason for me to *care* what context an object returned by 
some arbitrary method.  To me, that object's context and how it does 
lookups is its own private business and nothing to do with me.  Thus, most 
code in PEAK can be "context agnostic" when dealing with *other* objects, 
even if a particular object needs to know its *own* context in order to 
accomplish certain things.

What I'm curious about here, is whether we differ over a style issue here, 
or whether there are Z3 requirements that prevent you from using the same 
approach.  It may just be a style issue, in that you've never seemed 
particularly keen on encapsulation, while I on the other hand can be 
incredibly persnickety about it.  :)


>I don't think transparency has anything to do with this.  However,
>I think I may be getting a glimmer of what you're suggesting.
>
>I suppose you'd want the context to determine the policy for how to look
>up a component. So, for example, the implementation of getService might look
>something like:
>
>   def getService(context, servicename):
>       sm = getAdapter(context, IServiceManager)
>       return sm.getService(servicename)
>
>so that the context could determine the policy for looking up components.
>Of course, there's a recursion problem here. The adapter would need to be
>provided globally, via __conform__ or via having the context *be* an
>IServiceManager already.
>
>Is this the kind of thing you're talking about?

Nope.  My desire is to clarify responsibilities between objects and make 
context-driven coupling more explicit, instead of more implicit.


>Even with this flexability, I'm not sure how you'd use it. :)

I *do* implement it something like that in PEAK, but it actually iterates 
over an object's parents, asking them for whatever it's looking for and 
skipping those that don't know anything.  The useful bit about this is that 
objects don't have to be service managers in order to be part of a chain, 
they just have to know how to be part of the chain.  And that can be 
supplied by adapters, for many useful objects.  For example, a GUI window 
can consider its parent window to be its context.

Anyway, this doesn't speak to your current issue, so let me make a more 
concrete suggestion: why not wait till after containergeddon before you 
decide to use thread-local variables?  If context-awareness can be made 
sufficiently trivial, you might not *need* to do anything else.

I think one other factor that may be influencing the perceived complexity 
of context-awareness, is that right now context doesn't persist.  Perhaps 
many of the situations that now require a "viral" spread of 
context-awareness, are caused by the fact that the objects being passed 
around don't know what their "one true context" is.

Pretty much all context in PEAK comes from one of two places:

1) an object is given a context by its creator, and the context then stays 
with it until the object is dead.

2) an object is created by a factory of some kind which leaves it 
contextless until one of its consumers "suggests" a context for it.  It 
then keeps that context till it dies.

If containergeddon allows objects to keep track of their context, then I 
would think it would be possible to have the same principles apply.  Notice 
that when such an object is just being passed around by other objects, 
there is no need for them to manipulate or even care about what the 
object's context is, and so should stem the tide of "contagious 
ContextAwareness" that I understood to be motivating your proposal.