[Gsoc] Respector: A Grok Introspector Replacement

Martijn Faassen faassen at startifact.com
Fri Mar 28 09:49:22 EDT 2008


Hi Uli,

Some feedback from me.

On Thu, Mar 27, 2008 at 4:28 PM, Uli Fouquet <uli at gnufix.de> wrote:
[snip]
>  Problems with current approach
>  ==============================
>
>  The current approach of `docgrok`, the tool that does introspection
>  for Grok, suffers from some shortcomings:
>
>  - it is bound quite strictly to Grok, which means: it requires a full
>   fledged Grok instance running to work.

Of course any web-based UI on the ZODB will have to be bound to some
particular web application technology. I think that is fine - one
should be able to choose the web app technology you use in order to
build a web application. There's another dependency on Grok though
which is less good: if it's bound to Grok's particular way of doing
things to do its introspection. If it's depending on Grok-specific
assumptions to find its way through objects, it won't be able to show
information about non-Grok components. This situation can't be too bad
though, as we clearly are able to dig up information about non-Grok
Zope 3 packages already.

>  - it makes strong assumptions about the environment it runs in, namely
>   the existance of `apidoc` and other voluminous packages.

Again, just the existence of a few dependencies, voluminous or not, is
not a big problem - it can all be downloaded and installed. If apidoc
does way more or does the things in a very different way than desired,
it might not be the best tool for the job, but it'd be nicer to have a
stronger motivation to dump apidoc.

>  - it comes as a part of the grok core, which bloats the number of
>   packages needed to run grok, even if the introspecting functionality
>   is not wanted (for example in production environments).

+1 Yes, this is definitely something we need to take care of.

>  - it might be considered a security risk, because it frankly passes all
>   information to (authenticated) users without a possibility to
>   restrict access. There is no possibility to create something like a
>   'developer user'.

Yes, it'd be good if there were a form of user management, though it's
also tricky as it presupposes some form of authentication that may
conflict with what the application is doing.

>  - it does not provide an interface other than the Web-UI, nor does it
>   provide entry points for, say, remote debugging without a browser.

What other interface are you thinking about? I think it would be a
distraction to move away from a web UI to something else, though I can
imagine a clean Python API entry point.

>  - its API might be too complex. While it is possible to register
>   arbitrary types of things to be displayed by `docgrok`, the way to
>   do it might be plainer.

Making the API easier is a good goal. :)

>  - it makes too little use of other tools already available (basically
>   it wraps only `apidoc`).

Could you mention some of those other tools?

>  Apparently `docgrok` also lacks the ability to display important data
>  like directives used in a view, available layers, default values
>  etc. This, however, is more a question of diligence, because in
>  principle all this is supported by the current approach. It would only
>  need a concrete implementation for concrete types of things.

I wouldn't say that it's just a matter of "more work", but also of
"more thought". To display information about Grok directives used, it
might be worthwhile to refactor the way Grok directives work so that
their design is more introspectable. I've started working on this
inside of Martian.

Note that if you want to make this a more generic tool that is also
useful with Zope 3 and Zope 2, all knowledge of Grok related
information will need to go into plugins. I can imagine this
negatively impacting the user interface from a Grok perspective,
unless much care is taken to increase pluggability. Making this work
on Zope 2 will require a particular amount of work, as Grok on Zope 2
isn't there yet.

>  Renewing the Grok admin app
>  ===========================
[snip]
>  To cope with the deeper design flaws, I can think of a two-step
>  strategy:
>
>  1) factor out introspection stuff from the then standalone admin app
>    into an own package or subpackage that can be used also without
>    grok/grok.admin.

This is a good strategy to see the outlines of the underlying API that
is needed.

>  2) give it a new, more general, easy to use base.
>
>  All this should not be too complex, except the 'general base' stuff.

Could you describe what you mean by a general base? A new API that
retrieves the needed information that can then be used by the UI? Or
does it include some UI bits? Do you envision a generic UI that can be
plugged into for particular use cases, or specific UIs that all build
on a shared generic set of introspection APIs? It looks from the
following you're talking about the latter.

[snip]
>  - should separate the tasks of information retrieval and information
>   presentation. Nearly all tools existing in that area do that
>   internally, namely `docgrok`, `apidoc` and `zodbbrowser`. But they
>   all make assumptions about the used frontend (browser, wxpython-GUI)
>   and offer only support for that kind of 'display'.

I think we shouldn't worry too much about other front-ends like
wx-python. If there's a wx-python client already (which there is for
the ZODB-browser) then fine, but over the years with Zope I've become
rather skeptical of "fat client" approaches to Zope. People get
enthusiastic, and then not much happens. :) Web UIs are powerful
enough to take care of a lot these days anyway. And we're a web
developer community, so we know web UIs. That said, we *should* worry
about a clean API separation between UI and underlying library. This
library should be UI agnostic and just be a Python library. I don't
think you should pay special attention to alternative user interfaces
and building frameworks to support them, though, just an API that they
might use.

>   A well designed runtime-scanner might be prepared for collaboration
>   with unknown presentation 'plugins'.

I recommend against the idea of presentation plugins. I'd design this
as a library. Your presentation code then uses the library. I don't
see a plugin concept as necessary, as this overly complicates things
and may anticipate uses which will never materialize.

> The separation of retrieval and
>   presentation would ease the implementation of specialized frontends
>   for Plone, Grok, etc.

Yes, but use the word library and clean API please. Just have a
library with a clean API which tells me about stuff. Then on top of
this, you could build some higher level APIs. But at some point, hand
off the responsibility of how to display this stuff completely and let
the UI component pull this in in whatever fashion suits it. I imagine
you were already thinking alone these lines, but I just want to be
very explicit about it. :)

>  - might be prepared for modification of objects. This means, it might
>   provide a 'communication channel' to tell a frontend, whether a
>   certain attribute/object/thing is modifiable and perform this
>   modification on request.
>
>   This way, the scanner would turn into a debugging tool.

I like this idea. Needs careful API design again.

>  All this is _much_ work, probably too much for a single GSOC, but at
>  least a plain but well thought base implementation should be possible.
>
>  When the base API exists, numerous scanner extensions/frontends could
>  be implemented independant from each other later on.

I think it's important to include at least one frontend for the base
in any project to design a base. This way you really know what's
needed. :) The Grok frontend (which can be refactored into a base) is
a good candidate there.

[snip extra scanners for grok]

I think at least a view or viewlet scanner is generic enough to work
for Zope 3 and Zope 2 as well. Just the directive scanner is really
Grok specific.


>  The Grok admin UI needs a redesign anyway to address all the issues
>  mentioned above. So why not go back to the roots?
>
>  Furthermore this addresses a topic, which crosscuts several planned
>  projects for upcoming GSOC. Why not try a collaborative effort? Is there
>  a better opportunity?

I think we should be careful not to let projects depends on others too
much - the risk is just too high people will get stuck waiting for
each other. I think this risk can be minimized by:

* setting up a clear channel for communication between the involved
projects, students and mentors

* focusing on the development of Python libraries with limited
dependencies, and clean APIs, as the underpinnings of all projects.

I don't think we should hope for too much collaborative work on a
generic base beyond this. Since we have reusable Python libraries we
can pull them together relatively easily though.

Regards,

Martijn


More information about the Gsoc mailing list