[Zope3-dev] Context and Component Lookup

[Martijn Faassen]

Jim Fulton wrote:
> > I mean, could this
> >be different dependent on which component is found, or is this actually
> >not a worry as which components are found (that can find other components)
> >are under control of the calling context.
> >
> >Anyway, I'm still slightly concerned that the implicit behavior of
> >depending on the request's context is the default, instead of the
> >attempt to depend on the local context..
> 
> Attempting to depend on the local context is just as implicit.

All right, but..

> The default behavior is implicit in any case, that's why it's called
> "default". If something is implicit, I'd prefer something that is as
> predictable as possible. 

Agreed.

> If something doesn't have context, it's either
> going to get an error or it's going to use the global services. neither
> of these possibilities is appealing to me.

Speaking from a position where I don't have a set of use cases in
my head for such objects, getting an error seems more appealing to me.
Something that doesn't have a context explicitly given to it has no context 
and thus can't find anything. It's possible to give something a context
explicitly by employing a decorator.

I wouldn't even object to the case where these objects do have to implement
something to get to their context, as having a context seems to be an inherent 
part of using the CA. But apparently this is not desirable. 

> OTOH, if it does have context,
> it's going to use a local service that might also not be the one I want.

But at least the local service will have been set up to make the component
work by someone, and will have a greater chance of having been tested.
Interaction with the caller's context cannot be tested in advance.
The predictability here is that it behaves the same way *whatever* code
calls it. There's an encapsulation; there's less encapsulation if
things are going to be pulled from the caller's context (or possibly
even several callers back) -- which components are going to be needed
cannot be predicted without code inspection or extensive documentation,
or extensive experience with previous failures.

[snip]
> >It seems to me that whenever code manipulates objects in another container
> 
> You mean "site", not container.  A site defines configuration.  All of the
> objects in a site use the same configurations and components.

I expect many containers to be sites in that they have local configurations.
Don't you think that this will be the case? I also don't have any
idea whether a particular container *is* a (sub)site.

> >at all there's a chance of this happening, and I, the writer of calling 
> >the code, have no indication that this *cannot* happen. There may be a
> >site management folder placed there by someone and this can now cause
> >my code to break. Or if there's an existing one, any chance to it
> >can cause my code to break as I haven't supplied the right components.
> >
> >I seem to have no way to avoid this; the only case where this does not 
> >happen
> >if the other person has made sure to use the context=self pattern described
> >above in all component lookups; I'm not sure this is even possible as
> >some framework code may be doing this too..  At least in that case
> >it'll presumably work with a bunch of well known interfaces, though I'm
> >still concerned there may be a kind of 'spreading' effect as soon as one
> >object (implementing a well known interface) is put in the mix that does
> >component lookups of some less-common interface.
> 
> A component designer can specify and then must rely on it's component
> dependencies. (At some point, we need to come up with this specification
> scheme.)

A specification scheme can't be better than more encapsulation from
the caller's perspective.

>  It is the site assembler's job to make sure that the dependcies 
> are met.

The site assembler may be a whole bunch of people who place one or
two components to configure subsites. These may even be end users going
through a particular UI.

If I'm a site assembler and call a method on another site, it shouldn't
suddenly become my job to match the dependencies of *that* site *and*
any site that this site may be calling. You keep telling me
that:

  a) I should just bear with the responsibilities and unpredictability
     that calling into a (sub)site incurs 

or..if I try to express that this is a huge burden in some cases you tell
me:

  b) I shouldn't do that.

or alternatively

  c) the original author should depend on the local context in that case
     explicitly to avoid this problem. Fine, but I have no control over this  
     author.

calling methods on objects is not going to be a rare occasion! These
objects could be coming from all kinds of locations, including subsites.
I may be getting results of a catalog call, for instance, and may want
to loop through them calling methods on them.

If local context were used by the default, I could call any method on
any object I may get happily and know that if this object is in a
properly configured (sub)site my call is going to succeed. I also
know that this will happen no matter what code is doing the calling.

I feel this is lots better than the three unsatisfactory options
you keep presenting me with.

You seem to be driven by the use case of objects doing CA lookups while
they have no location of their own nor are explicitly given a location
of their own. Can't we call YAGNI on this for a while?

> If I get a bunch of electronic components and assemble them 
> incorrectly,
> the components will not perform their functions correctly. This isn't the
> fault of the component designers.

Calling a method on an object is *not* assembling a component. 

The analogy is that I assemble electronic components correctly and install it 
somewhere, and it works. Then you come along and assemble your own
electronic components and wire them to mine.

Suddenly you notice that your assembly of electronic components is 
incomplete, and you have to install a lot of components yourself that
already existed in my assembly. Then you have to test it.

My question is what use was it for me to assemble those components
in the first place? You seem to have to replicate most of the work
I did as soon as you want to use my assembly.

> I suggest that whenever an application shares components accross sites, it's
> the responsibility of the application assembler to make sure than the shared
> components have a supportive environment whereever they are used. This is a
> burden on the application assembler that is, hopefully, justified by the 
> benefits.
> If it isn't than such sharing should be avoided.

You seem to have a very different idea of the granularity of sites
than I do. I consider subsites to be a very regular occurance, and
a way to locally modify behavior. This modification will quite
regularly done by site administrators or even end users.

Imagine a case that where you add a container, a default object is
automatically added to it as its 'index.html'. Now in my subsite
I configure this default object to be an instance of some HTMLPage.
In your subsite you configure this object to be a GeeWhizFlashPage.

Now I'm writing a method that automatically adds a default object
to all containers that don't have it yet, for a given site.

Is this a case of sharing components across subsites?

> I suggest that the most common case of sharing is the case of nested sites
> where the sub-site overrides configuration of the containing sites, An
> object from the super-site will use customized components when used in the
> subsite.

Yes, exactly as described. So how does this work if the configuration
used is determined by the location of the request?
Or are you suggesting that in this case all lookups should take place
explicitly in their own locations?

> >>Only if the client cares about the storage.  It isn't not in the 
> >>interface,
> >>the client probably doesn't care. If it does care, the client should
> >>be using a different interface.
> >
> >The client doesn't care about storage, it just assumes it's using
> >the *same* storage as another client.
> 
> In this case, the client cares very much about the storage. Note that
> it doesn't care about the type of storage at all. ZODB or RDBMS make
> no difference. The cleint wants to use the same storage instance.

No, the client doesn't care at all about what storage is used. The
client just cares that if it calls a method on an object somewhere,
the result for a given set of arguments will be the same than if
another method called that method on that same object given that
the system starts in the same state.

Underlying the method that is called, a storage implementation is
used. This is no business of the client. The client is just
calling a method and couldn't care less where this information is
stored, or whether it's generated, or whatever else may be the case.

If client conceptually sets a value to be X using a setter method, then a 
reasonable expectation is that *another* client accessing this value through
an accessor method will get X out.

If the way storage works is dependent on the calling method, this invariant
can be broken.

You can't say that this client method cares very much about the storage; 
the clients just expects the interface it's calling into to
behave sanely. To me it is part of the contract of such an interface to
maintain state this way.

> If the client really wants to maintain that invariant, it should really
> hold a refernce to the storage.

What an incredibly onorous burden to saddle the poor calling method with.
I am having huge trouble understanding where you're coming from here..
 
> > If it has to supply it in the
> >context itself, it can never be sure.
> 
> I'm not esure what all of these "it"s are, but I'll note that if a
> client has a contract that it always uses the storage for its site,
> then it can satisfy this by being context aware and providing itself as
> the context for component lookup.

How can the client do this? It's just calling a method somewhere on
some object, which may or may not be doing a component lookup as
a result. The client can never be sure it's actually supplying the
same storage components than some other client. 

> >Well, if you had the kerosene fridge in your context, and your wife has
> >the electric one in her context, and you're putting stuff in and she
> >can't find them, this is rather odd, innit? :)
> 
> As noted above, this has nothing to do with storage type.  If I always
> want to use the fridge in my house, then I need to pass the house
> to the fridge lookup routine. This is easy enough to do. 

The more I think about it the more convinced I become that this behavior,
passing the location to lookup code, should be the default behavior.
Only if one wants to use the request's context should one get 
explicit. One is likely in a rather weird object that has no way to
get to any location in such a case.

> If I always
> want to use the same fridge no matter what, then I'd better keep a reference
> to it.

Client X from site A has a hard reference to the fridge.

Client Y from site B has a hard reference to the fridge too.

Client X can put stuff in the fridge that client Y can't see. There is
no way for client X to guarantee what happens when client Y uses
the fridge. This is because the fridge does lookups to IFridgeStorage,
and X and Y supply their own IFridgeStorage
components each, and don't know about each other's existence (and
shouldn't, they only share the fridge).

If the fridge looked up IFridgeStorage in its own location, client X
and Y would have no problem using the fridge, and don't have to worry
about supplying IFridgeStorage components either.

> >If it were supplying its own environment I would not have to meet its
> >needs in order to use it.
> 
> This corresponds to the first case above.  In the proposal, an object
> certainly has the option of providing it's own environment.

I'm having a problem with 'option'. The optional behavior seems to me
the most reasonable behavior.

[snip tucking away stuff in other sites and expecting other code to
get it out again]
> Frankly, this is a use case that doesn't seem very important to me.

To me it's a basic invariant that's being broken, so this is where
our opinion differs.

> You seem to want an object to *always* use it's own environment.

Yes, by default.

> This is unaacceptable to me because:
> 
> - Many object's don't keep track of their environment

If objects don't keep track of their environment and nothing else is
doing their tracking for them, they have no business looking up stuff
in any environment.

So, you introduce the request context to be the environment of all
objects, even those that track their own environment just fine.
This is a rather aggressive default which I've have been trying to
show breaks important expectations of programmers. Whenever an
object comes into my client code, I need to worry what components
this object may want to look up if I just want to call a method on it.

> - In most cases, I want components to adapt to their envoronment by
>   using the components provided their.

Why do you want components to adapt to the environment of the initially
calling code in most cases? I still can't come up with cases where this
is sensible behavior outside possibly views.

> I'm willing to place the burden
>   on the application assemler to make sure that the necessary components
>   are in place to support cross-site sharing if that's something the
>   application wants to do.

I've been trying to show that this burden is unreasonable, and that
cross-site sharing of components is common in the case of subsites.

>   I acknowledge that there *are* cases where an object must use components
>   from it's site. When such constraints exist, there is an explicit way
>   to satisfy them.

My position is that these cases are common and the road of the least
surprise, best encapsulation and lesser burden.

[snip]
> >If the context=self pattern is used a lot it might be done, but 
> >if you're just *assembling* components you have no control over this.
> >And if it's not used, the component assembler cannot decide for
> >instance that a certain service uses a RDB backend by plugging in the
> >right component for this.
> 
> In the example you have, the type of storage doesn't matter.  The invariant
> you have way that an object always used the same storage, regardless of 
> type.

In the previous example this was the case. This is a new example.

> This is a very specific requiremnt that should be met by establishing a 
> direct
> association between the components or by writing the component to always
> use the storage from it's site.

Yes, if you're a programmer that can change the code.

If you're a component assembler you do not have that ability, and thus no way
to say, guarantee that all user data is stored in a relational backend,
except for a particular subsite where I want it to be stored in the
ZODB, unless all components I use do lookup in their local context.

> > Unless the component assembler has complete
> >control over the whole system,
> 
> They generally do.

Then your vision of what the component assembler does is rather radically
different from mine. See previous use cases of local configuration by
site managers or end users.

> > that is, but presumably we have different
> >sites in a system in part for the reason that there may be multiple
> >independent component assemblers (still working together).
> 
> If you truly have independent assemblers, then sharing content accross
> systems is going to depend on some coordination.

Yes, and this coordination is actually satisfying all the requirements of
the different site.

Anyway, I don't want truly independent ones, I want multiple component
assemblers that can expect that their subsite works well without
having to worry a lot about what may be going on elsewhere in the site
where they cooperate.

> If adequate coordination
> isn't possible, then sharing is not a good idea.

Yes, my position is that the proposed behavior makes adequate coordination
much harder, and sharing becomes therefore much harder. This is your
"don't do that then" position. :) I will complain that I do want to
do this, and your answer here would be again that
"well then change the code to do local lookups/bind tightly", whereas
I will complain that local lookups should be the default anyway. :)

[snip]
> >I don't understand here what you are referring to by saying 'doesn't it
> >already have one'? One context? I thought was is not possible for this
> >other assemblage to use its own context when I call into it from
> >another site, at least not without altering its code.
> 
> If I were hooking component assemblages together, I would hook them together
> in the same site.  In this case, everything in the original assemblages is
> in the same site and this available.

(but what about subsites?)

This will make code that does stuff with multiple sites rather hard
to implement; I can imagine various system administration tools like this
(give me the title of all the roots of all the sites, say).

Simple container views also need to do this..looks like any container that
can contain sites needs to actually have the component assembly of the
contained sites in itself in order to safely call methods on the contained
objects.

> >>You'd be typically building a
> >>single site by composing these and other assemblages.
> >
> >I'm not sure what this means. Where are these assemblages coming from?
> >We have one site, so one assemblage?
> 
> No, a site can be built from multiple assemblages.

Where are these assemblages sitting? The assemblages don't really function
as black boxes, right, in that at any point they may look up any component
in some sibling assemblage. 

> > It seems to
> >me such a data dependency is fairly easily introduced  and supplying
> >the context to be self is easily forgotten. When testing the thing
> >even works. It's just that when you reuse X from somewhere else that
> >it starts failing.
> 
> Perhaps this is a danger.
> 
> OTOH, the issues that I've raised need solutions:
> 
> - Not all objects track context.  When these objects access the component
>   architecture, the results they get should reflect site-specific
>   customizations.

Perhaps we need a few specific use cases and specific objects here
as examples. Can we have some examples of objects that:

  * don't track context themselves, for instance by being contained

  * don't have their context tracked for them by some intimately related object
    that decorates them when necessary

  * can't have the context passed to them explicitly either (so that they
    can do component lookups on the context object instead of on themselves)

> - Sometimes objects will be used in a site that is related to but different
>   than the site they live in. In most cases, the components used with these 
>   objects
>   should be chosen from the host site.  This seems obvious in the case of
>   presentation components.

I agree that this seems to be usually the case for presentation components.
So perhaps the presentation layer by default looks in the request for
context. Perhaps this is a layer slightly above the underlying lookup
mechanisms, and this layer takes care of supplying the request as the
context (and this can be explicitly overridden during lookups if
desired).

> I don't think it usually applies to non-presentation components as well.

Yes, the problems I see exist for non-presentation components. 
So perhaps I can add another criteria to the object described above:

  * what is being looked up for this object is not a presentation component.

Wild speculation:

Hm, perhaps getView should read like this:

def getView(request, name, object, context=None):

where request is the default context where lookups take place. We get
views for a request, not for an object, in a way.

Regards,

Martijn