[Zope3-dev] Interface declaration API

[Phillip J. Eby]

At 04:35 PM 3/11/03 -0500, Jim Fulton wrote:
>Phillip J. Eby wrote:
>>At 02:46 PM 3/11/03 -0500, Jim Fulton wrote:
>>
>>>Subclassing is done to reuse implementation. If interfaces were
>>>inherited, then you would be unable to subclass a class without
>>>also promising to implement the base class interfaces.  I've always
>>>viewed this as an undesireable requirement, but maybe I'm wrong,
>>>since so many people desire it, ;) So I'm willing to give on this.
>>
>>Ironically, you then go on to say that 'doesNotImplement()' would be a 
>>YAGNI.  :)  I actually suggested 'doesNotImplement()' as a way to deal 
>>with the situation above.  If one wishes to reject base class interfaces, 
>>one might then do so explicitly.
>
>The way you would reject base-class interfaces is to use implements. That is,
>if you use implements, you implement *just* what you say you implement,
>so I still think 'doesNotImplement()' is a yagni.

This is probably a matter of design principles.  In general, I go for the 
Liskov substitutability principle; that is, I should be able to use a 
subclass where the superclass is acceptable.  This implies, in other words, 
that failing to implement a superclass' interfaces is a no-no.  In the rare 
case where I want to reject the legacy of my ancestors, I want to 
explicitly identify what contracts I'm rejecting, rather than try to 
reiterate every contract I might still implicitly be supporting.  I only 
want to use 'implements()' if I want to reject *all* ancestral contracts, 
which is something I've never wanted to do.  For that matter, because of 
the LSP, I'm not sure I've ever actually rejected even one ancestral 
contract, but I can easily conceive of situations where I might want to 
reject some subset of them.

I will concede that:

class A(B,C,D,E):

    implements( implementedBy(B,C,D), IE1, IE2 )

is almost as usable as:

class A(B,C,D,E):
     doesNotImplement(IE3)

where IE1, IE2, and IE3 are contracts of E, and I want class A to reject 
IE3.  I find it annoying on principle though, since I want the rejection to 
be more visible.  Violation of the LSP may indicate a "suspicious smell" in 
the inheriting class.  (For example, that it is using inheritance when 
delegation may be a more appropriate technique.)  Not only does the 
explicit 'implements()' give no clue about this "smell", it also creates a 
possible super-to-subclass maintenance dependency that may be bad for 
application frameworks where user code inherits from framework-supplied 
base classes.



>>>Soooooooo ....
>>>
>>>Let's suppose we decided that in:
>>>
>>>   class C(A, B):
>>>
>>>      alsoImplements(IFoo)
>>>
>>>The outcome would be that C's instances would implement
>>>IFoo and whatever A's and B's instances implement.
>>>
>>>This is implementable, but with a hack. The hack is that the
>>>descriptor needs to update itself the first time that it's used. :(
>>>
>>>In::
>>>
>>>
>>>   class C(A, B):
>>>
>>>      pass
>>>
>>>The outcome would be that C's instances would implement
>>>whatever A's and B's instances implement.
>>>
>>>This too requires a pretty heavy hack.  The first time you tried to
>>>get the instance interface spec for a C, you'd inherit a spec from
>>>A or B. The inherited descriptor would detect that it was inherited
>>>and create a new descriptor for C, merging the interfaces from A and
>>>B.
>>>
>>>Finally, with:
>>>
>>>   class C(A, B):
>>>
>>>      implements(IFoo)
>>>
>>>The outcome would be that C's instances would implement
>>>only IFoo.
>>
>>I've previously implemented late-binding class descriptors of this kind, 
>>and I agree that they aren't appropriate for Zope 3.  One potential 
>>issue, for example, is thread-safety of the object that does the late 
>>binding (because non-persistent classes are normally shared between threads).
>>I do notice, however, that your description seems to imply a 
>>descriptor-based implementation.
>
>That's what I was thinking of. Hm. I'll have to ponder a non-descriptor
>implementation. It might make things cleaner if we decide change the 
>inheritence
>semantics.  I was thinking in terms of descriptors because they avoid having
>to sniff to decide if something is a class, which is *really really* painful
>in the presense of security proxies.

I'm not sure why you need this.  Consider:

def directlyProvidedBy(ob):
     if '__provides__' in ob.__dict__:
         return ob.__provides__
     spec = ob.__provides__ = InterfaceSpecification()
     return spec

def implementedBy(klass):
     if '__implements__' in klass.__dict__:
         return klass.__implements__
     spec = klass.__implements__ = InterfaceSpecification() # XXX copy or 
merge from __bases__
     return spec

def providedBy(ob):
     return directlyProvidedBy(ob) + implementedBy(ob.__class__)

This should work fine for "plain" objects, classes, metaclasses, and 
"turtles all the way up".  :)  The only sticky bit is the presence of 
'__dict__'; but if something's __class__ doesn't have a __dict__, it's not 
going to be something whose ultimate metatype is 'type'.  So the above does 
need some try/except handling for a missing __dict__ attribute.  The except 
block would need to check if the non-dicted thing's __class__ has a "data 
descriptor" for that attribute name, and if so, use it.  This would allow 
__slots__ to be usable for the __provides__ and __implements__ attributes.

AFAICT, no part of this approach either requires or prohibits the use of 
descriptors.  But the __dict__ check *has* to take place (no matter how you 
implement this, with descriptors or not!) because you otherwise can't 
distinguish between an explicit and an implicit attribute value, when you 
don't know if what you're dealing with is a "class" in some sense.


>   I guess we shouldn't get too hung up
>on implementation difficulties at this point and focus on what we want.

Okay, if we're not worrying about implementation, then what I suggest we 
want is to inherit the union of the interfaces declared by a class' 
__bases__.  :)


> > If the intention is to encapsulate the
>>implementation, it's not clear to me that putting the behavior in a
>>descriptor is the most useful way to go.  For example, 'implementedBy()' 
>>might look something like this:
>>def implementedBy(klass):
>>     spec = klass.__dict__.get('__implements__')
>>     if spec is None:
>>         spec = klass.__implements__ = InterfaceSpecification(
>>             *tuple([implementedBy(c) for c in klass.__bases__])
>>         )
>>     return spec
>
>where this is a non-descriptor-based approach.

Yep.


> > Also, you have to
>>have *something* that does this, if you want 'implementedBy()' to return 
>>a mutable object!
>
>Is your point that you have to mutate the class if you want 'implementedBy()'
>to return a mutable object? You are right, although you could arrange
>for the mutation to be defered until you actually modified the spec, which
>wouldn't be so bad.

Maybe.  Threadsafety is still at issue.  Complexity and performance 
too.  You now have to return a new object from the descriptor's return 
value, that knows where you were retrieving it from (like a context 
wrapper) so that it can mutate the correct thing.  That seems like a cure 
worse than the disease.  It would be much simpler to mutate-on-query and be 
done with it.

Anyway, I'm not objecting to the complexity, per se, just saying that 
__mro__-first inheritance of declaration doesn't actually buy you any real 
simplification.


>>So, if you really want to simplify this, I'd suggest going back to 
>>immutable interface specs.
>
>I don't see how that simplified anything. Is your point that you want to
>avoid having a query method modify the thing being queried?

If you use immutable specs, combined with __mro__-first inheritance, and 
mutator API's, you don't have to write any code to handle the special case 
of an inherited interface specficiation.


> > This would then require an explicit
>>rebinding (e.g. klass.__implements__ += IFoo)
>
>That would be unacceptable, as it would expose the declaration implementation,
>which we want to avoid.

I meant that such rebinding would be wrapped in an API (addImplements(), 
setImplements(), etc.).


> > and wouldn't need a check
>>for whether the specs were inherited or not.  You also wouldn't need a 
>>descriptor, which means that "classic" classes might be better 
>>supported.  (I'm not positive that any of classic classes' limitations on 
>>descriptors are an issue here, though.)
>
>Is your objection that:
>
>   getImplemented(cls)
>
>could mutate the class?
>
>What if it didn't, but:
>
>   getImplemented(cls).set(I)
>
>did?

I don't object to either.  I'm just saying that avoiding __bases__-merged 
inheritance doesn't simplify the implementation any.  If you have mutable 
descriptors, you need inheritance checking, so the extra cost of using 
__bases__-merged inheritance doesn't add any significant complexity.


>>In order to preserve encapsulation, you would need to have functions 
>>like  'addImplements()', 'addProvides()', 'setImplements()', etc.  While 
>>this expands the number of API functions, it also increases the 
>>explicitness of such operations (one may grep for 'setImplements()' more 
>>easily than for 'foo=implementedBy(Bar) ...  multiple lines of code ... 
>>foo.add(IBaz)'.  That is, one is now required to put the operation in the 
>>same place with the thing being operated upon.
>>Doing this would actually give you an implementation benefit to sticking 
>>with "first in MRO" inheritance semantics for interface declarations, er, 
>>specifications.
>
>how so?

Because then you don't need any code that checks for whether the value is 
inherited.  API functions can use the equivalent of 
'getattr(ob,"__implements__",())' and not worry about whether it's being 
inherited, and use ob.__implements__ = whatever in order to establish a new 
specification.


>>(By the way...  I always stumble on calling them "interface 
>>specifications".  To me, the interface specification is the "class 
>>IFoo(Interface):" block, while the "__implements__ = IFoo" is an 
>>*interface declaration*.  But I may be alone in this.)
>
>I see your point.
>
>>Anyway...  what I'm saying is, if you stick with mutable specifications, 
>>you might as well implement "merged __bases__" inheritance, because there 
>>is no less implementation complexity.
>
>Because you have to mutate classes in either case?

Right.


> > AFAICT, to reduce the
>>implementation complexity from what you described, you have to also make 
>>specifications immutable, so that it never matters if the specification 
>>is inherited or not.  Otherwise, you are right back in the same boat of 
>>needing to either have a lazily-binding descriptor, or a function that 
>>acts like one "from the outside".
>
>
>Even if they were immutable, you'd want to manipulate the descriptor 
>and/or the class
>for efficiency sake if you don't want the current inheritence semantics.

Yes, but that's not what I was suggesting.  What I was suggesting was that 
if implementation complexity was really a good reason to use __mro__-first 
inheritance, then you should also drop mutable specs, since that would 
allow you to greatly simplify the implementation and reduce the 
threadsafety concern to being an issue only when modifying a class' 
interface specification.

OTOH, if mutable specs are desirable enough to override the complexity 
concerns (personally, I don't see that being the case; in fact I personally 
think that mutable specs are to some extent inherently undesirable) then it 
is no big deal to add __bases__-merging inheritance, 'alsoImplements()', 
and 'doesNotImplement()'; they have only O(1) impact on performance and 
negligible impact on the overall code complexity.  Most of the 
implementation complexity, IMO, will come from supporting mutable specs, 
not the rest.

In short: Since I've offered to back up my desires with code, I could go 
for either implementing immutable specs and __mro__-first inheritance 
(simple way), or adding my fancy bits (__bases__, alsoImplements, 
doesNotImplement) to a mutable specs approach.  If you guys are going to do 
the hard work to make mutable specs workable (and again, I don't see why 
they're valuable), then I don't see why I shouldn't benefit by contributing 
a few extra bells and whistles.  :)  Conversely, if you don't want me to 
contribute the bells and whistles, I don't see why you shouldn't just do 
the "simplest thing that could work" and do immutable specs + __mro__-first 
inheritance.

I could be happy with either approach, speaking both as a consumer and 
potential co-implementer.