[Zope3-dev] Re: Interface declaration API

[Jim Fulton]

Phillip J. Eby wrote:

...

>> The problem is that, in the presense of security proxies, you don't 
>> have access
>> to the __dict__ attribute. All you have access to is attributes.
>>
>> Descriptors have a major benefit, in the presense of security proxies, 
>> which is
>> that, within the descriptor, you get access to an unproxied object.
>> But, I realized since yesterday that the descriptor need not actually 
>> store all
>> of the information.
> 
> 
> It's true that the descriptor needn't store everything.  In fact, it 
> needn't store *anything*.  You could write descriptors that did the 
> __dict__ check on an attribute of another name.  However, this wouldn't 
> be helpful if the class doesn't *have* this descriptor.

That's a special case that has to be handled, but that special
case is handled through a simpler attribute access, which You'll do
to get the descriptor anyway.

> 
>>  It can walk class bases if it wan't to.  So, I may actually
>> be able to avoid some of the weird hacks, unless I can't.
>>
>> It's time for me to start prototyping things.
> 
> 
> I can save you (some) trouble by pointing out that descriptors will not 
> help you in the situation where the class or instance never declares any 
> interfaces at all:
> 
> class Foo: pass
> 
> foo = Foo()
> 
> providedBy(foo)    # will fail without ability to check the __dict__

It won't fail because it will get an attribute error and return an
empty interface spec.  Of course, at that point, mutation will be difficult.
Mutation won't, in general, be possible through security proxies.


> Descriptors are only helpful if they're *there*.  If you need to have 
> modify-on-query (or even delayed modification), you'll need to be able 
> to *insert* the descriptor into the class.

Yes, at which point we will.

> With __dict__ manipulation, you can do this.  With a bit of trickery, 
> you can even do it to classes -- even built-in types.  This is a nice 
> extra benefit.

You can't get at the __dict__ of proxied objects.  Querying operations
will need to be be able to get at the __dict__ or do other introspection
that would be difficult or impossible without descriptors, which allow us
to get past the proxy.  As I mentioned above, we probably won't allow
mutation through proxies.

> With descriptors only, you are powerless to do this to anything that 
> doesn't have the descriptor already present.

When you need one, you can add one. For example:

   classInstanceInterfaces(Foo).set(IFoo)

would add a descriptor to Foo.

> Last, but not least, attribute access has some tricky semantics.  You 
> may be able to rule these out as not affecting what you want to do, but 
> given an object 'ob':
> 
> 1) An attribute may be defined by 'ob.attr' (in instance dict or slot)
> 2) An attribute may be defined by 'ob.__class__' or its bases
> 3) An attribute may be defined by 'ob.__bases__' if ob is a class
> 
> If you don't want to special-case for whether 'ob' is a class, you can't 
> use bare attribute access in the absence of a descriptor.

Right, that's where the descriptor helps out, because it makes the
special casing sane.  In a descripor, we can trivially tell whether
we are being called on a class or an instance.  The hard part of the
special casing has been done for us. We will use different logic
depending on whether a descriptor is used on a class or on an instance.

> Also, descriptors have another peculiarity...  if you define a "data 
> descriptor" (i.e. one that implements __set__), then the precedence 
> rules for attribute access are *different*.

Yes. I know.

 > Data descriptors take
> precedence over what's in __dict__, and this can make metaclass access 
> "weird".  Example:
> 
> class Meta(type):
>    aProp = property(...)
> 
> class Class(object):
>     __metaclass__ = Meta
>     aProp = property(...)
> 
> ob = Class()
> 
> In the above code, accessing 'ob.aProp' will call the __get__ of 
> Class.__dict__['aProp'], but accessing 'Class.aProp' will call the 
> __get__ of Meta.__dict__['aProp'].  This means that you can't just write 
> a '__get__' that looks to see whether it's being called on the instance 
> or the class in that case.

Why not?  In both of the cases you've mentioned, we know we are being called
on the "instance" as far as the descriptor is concerned.

> Again, I don't know if this will impact what you have in mind; it's hazy 
> to me right now whether you plan to use descriptors with __set__ or just 
> __get__.  But you should be aware of this because it impacts interface 
> declaration in the presence of metaclasses, if you use descriptors.

I am aware of these issues and will take them into account.

> 
> 
>> Maybe. I think that this framework is important enough that we
>> shouldn't get too hung up on complexity. Performance is important, *but*,
>> I've found that to get reasonable performance, you really need to employ
>> caching in ways that make the actual algorithm performance less 
>> important.
> 
> 
> Absolutely; the PEAK framework makes heavy use of a primitive for 
> defining attributes based on a lazy, cached, one-time computation.  
> (Think of a computed attribute that is computed only once.)  I already 
> have code for all that in C (Pyrex, actually) that can even deal with 
> all the metaclass funkiness and threadsafety.  So I have a pretty good 
> idea of what the complexity is.
> 
> But, the algorithm for determining *whether* you can use the cached 
> result is critical to performance, since it must be executed on every 
> query.

Yup.  The selection of the cache key is critical.

> 
>> For we'll need to cache isImplementedBy calls, making the efficiencey
>> of the uncached computation less important.
> 
> 
> Actually, by having InterfaceSpecification instance simply keep a set or 
> dictionary of the declared interfaces and have a __contains__ method 
> that delegates to the dictionary, you wouldn't need an explicit cache:
> 
> def isImplementedBy(self,ob):
>     return self in implementedBy(ob.__class__) or self in 
> directlyProvidedBy(ob)

That doesn't work, because you need to do extends checks, not just sameness
checks.

Consider:

   class I1(Interface): pass
   class I2(I1): pass

   class C: implements(I2)

   I1.isImplementedBy(C())

The answer needs to be true, even though I1 isn't in the interface declaration
for I2.

Of course, the interface specification could include a set of the interfaces
in the specification and all of the interfaces that they extend.

> 
>> Let's put the implementation issues aside.
> 
> 
> Oops.  :)  I'm going to send this message anyway, because I think you 
> should know about some of the peculiarities of descriptors.  I've spent 
> a *lot* of time learning about them the hard way.

Thanks. I was aware of those issues. I've done a lot of work with descripotors
myself, although I've tried to steer clear of meta classes.

Jim