[Zope3-dev] ZConfig schema extensibility q's

[Phillip J. Eby]

At 05:01 PM 9/23/03 -0400, Fred L. Drake, Jr. wrote:

>Phillip J. Eby writes:
>  > There are no *unit* tests.  Another developer at my shop used it in a
>  > *functional* test.  :)
>
>So unit tests are certain to either find bugs or be incomplete.  ;-)

Well, if you want to insist on being XP-correct, it is quite proper to 
write experimental prototype code in order to better define the 
requirements, so you can write good unit tests.  So, I "spiked".  :)


>  > Urgh.  I'm not keen on this.  How about, you don't have to specify it
>  > unless there is more than one datatype specified by the immediate
>  > bases?  IOW, specification would be required for disambiguation.
>
>Nope; the base datatypes would have to all be the same to "inherit"
>it, otherwise something is lost.  But most reasonable apps will just
>use the "null" datatype for the schema, so that should not be a big
>deal in practice.

Whaaaa???  I've never used the null datatype for the schema, and I can't 
imagine ever wanting to.  But if I did, why couldn't I just say 
datatype="null" or datatype="ZConfig.sometype"?



>An unspecified datatype is equivalent to a specification of "null" for
>schema; doing what you describe means that "null" isn't really a
>specification for datatype anymore.  This will bite someone in a
>mysterious way; better to require explicit control.

I'm baffled here.  If you're *not* extending another schema, of course no 
datatype is no datatype.  If you *are* extending a schema, then no datatype 
means inherit the datatype.  I don't understand why you find this 
confusing.  It's no different, IMO, than the code:

class A:
    dataype = "something"

class B(A):
    pass

class C(A):
    datatype = "another thing"

If you need to be able to turn a datatype *back* into a null in an extended 
schema, why not just have some datatype like "ZConfig-Section" available as 
a standard datatype?  Then you say, "if no datatype is defined or 
inherited, the default datatype is ZConfig-Section".



>  > * keytype and datatype must be specified, *unless* all of the listed base
>  > schemas refer to the same type.  (IOW, if you're doing single 
> inheritance,
>  > you shouldn't need to specify them.)
>
>For keytype: agreed.
>For datatype: see comments above.

I've got to say I'm -1 on requiring an extender to specify a datatype; it 
runs counter to my original use case for extending schemas in the first place.

In PEAK, there is a base schema that's intended to be used for creating 
event-driven applications (that use a Twisted reactor or PEAK's 
mini-reactor).  It defines some top-level parameters to control the 
application as a whole, and expects to have sections included that 
implement certain abstract types for event-driven components.

Now, the only way that schema is usable to anybody is if they extend it, 
because they can't put their own types in it otherwise.  Without schema 
extension, they have to copy all the key and section definitions, as well 
as the schema tag.  With my proposed approach, they need only do:

<schema extends="pkgfile:peak.running/EventDriven.xml">
...whatever
</schema>

in order to use your approach they would need to do *this*:

<schema extends="pkgfile:peak.running/EventDriven.xml"
         datatype="peak.running.commands.EventDriven.fromZConfig"
 >
...whatever
</schema>

The latter is encoding detailed implementation information that the user of 
this schema neither needs nor cares about.  He or she is (more likely than 
not) using the EventDriven schema's implementation as-is, because the 
value-add is in their custom plug-ins to the top-level object.  If I change 
implementation details of EventDriven.xml, users should not need to go back 
to all their schemas to change the datatype.  That seems wrong to me, as 
it's needlessly exposing an implementation detail.

(Btw, in case you're wondering what 'pkgfile' is, it's a PEAK-defined URL 
scheme for files in Python packages; PEAK uses resource loader subclasses 
that use PEAK's URL retrieval system (which calls down to urllib for most 
of the schemes that ZConfig files would use.)


>  >>> Multiple URLs can be supplied to the "extends" attribute, separated
>  >>> by whitespace.
>  >>
>  >> Should the same treatment apply to sectiontypes?
>  >
>  > Eh?  I don't understand.
>
>Section types can extend another section type; should they support
>multiple "bases" as well?

I haven't run into that as an issue, so I'm +0.


>  > Hm.  Interesting possibility, but it could take a lot more work to
>  > implement.  In essence, you might need to *modify* the existing 
> definitions
>  > of those items in-place, because other types might have references to
>  > them.  Anyway, I'm not currently being bothered by that limitation.  The
>  > other items discussed above seem much more valuable in the short term.
>
>No; I think you're thinking about section *types*, not section and key
>instances.

You're right; keys and sections are only referenced by their type, so 
they'd be a lot simpler to update.  So I'm more + on it, but not all the 
way to +1, since I haven't encountered the problem yet.  :)


>  > Or, to put it another way, until we have some kind of schema 
> extensibility,
>  > nobody will be able to *have* the problem that you're talking about, 
> yet.  :)
>
>I've got a description of an %import keyword to allow the equivalent
>to <import package='...'/> from the config file syntax, and Shane has
>indicated in discussion that it sounds like a good thing.  (It would
>be useful with his Ape, and allow Ape to be used without changing
>zopeschema.xml; it could also be used for things like
>DirectoryStorage.)  I'll post my description of that later (intending
>that to mean "later *today*").

Hm.  That would work, too, as it would mean no need to even extend a schema 
just to hack some section types in.  On the downside, it unnecessarily 
exposes an implementation detail to the *end user* of the config file, 
who's now required to put the %import in there.  If the end user is the one 
doing the extending, it's great, but for my situation, the person who 
created the extension should take care of this.

I can definitely see the benefit of %import for using an existing 
application's schema, to extend the application with custom code.  But, 
that's not my use case, which is to allow extending the application itself, 
thus creating a new application, which is then given to an end user to 
configure.