[Grok-dev] Re: ANN: New Grok Powered Website
kevin at mcweekly.com
Mon Feb 25 12:37:00 EST 2008
Martijn Faassen wrote:
> Thanks for letting us know about this website! Once every while I
> wondered how you were doing as you were rather quiet on the list, so I
> am very happy to see that the site is up and running.
> Kevin Smith wrote:
>> Biggest hurdles: lack of export/import, moving large data sets,
> I'd be nice if you could expand on this: what kind of import/export
> system would you need? (what would be sufficient, what would be
> ideal?) How did moving large data sets cause problems?
I shouldn't have included moving large data sets as that had nothing to
do with Grok/Zope3. Large data sets are just unwieldy.
As far as import/export is concerned, I was very happy with Zope2's
.zexp files. AFAICT the only thing that prevents the ZODB built-in
import/export mechanism from working is that it pickles __parent__ which
means exporting *anything* exports the *entire* database.
I actually had basic pickling working for awhile but didn't have the
time/know-how to isolate the issue to see if an upgrade to Grok/Zope3 or
programming on my part broke it.
I did see stumble across your "secret" work in Zope SVN on an export
alternative, but from what I saw took more manual configuration than I
had time to implement.
>> lack of docs/examples (at the time),
> Can still be much improved, but we've been moving forward on this indeed.
>> lack of layers and viewlets,
> Thank you here for the contributions! Layers have landed, and viewlets
> will do so soon enough. You should thank Tim for pushing me at the
> Snow Sprint into looking into them - I'm quite pleased with the
> progress we made there.
Thank you Tim!
>> Catalog scaling limitations
> This is interesting to find out more about. Any ideas as to how to
> approach a solution? A common way that people have approached this
> problem is to use a relational database (or custom index software such
> as lucene) to back up the catalog index. I think it wouldn't be very
> hard to come up with a set of indexes that are RDB backed. Harder, but
> possible would be to make hurry.query aware of them so that it create
> smart SQL queries.
My solution was to create a new Catalog for each year's worth of
content. For my use case this works. I didn't want to add the additional
overhead of real RDB interactions and lucene which may make a more
>> lack of respect to how long it takes to build a custom CMS from
>> ground up
> Already discussed elsewhere in the thread; I'm glad you are happy now
> we have our dependencies under control.
>> Biggest helpers: GROK SMASH ZCML, grok.View, hurry.query, groklets, a
>> renewed respect for ZPT's especially macros, Martijn's sense of
>> humor, Philipp's book, emacs, screen, open source in general,
> I'm glad you think I got a sense of humor. :)
> [moving this bit from the bottom so I can comment on it in a better
> > Grok 0.9 (heavily expanded to use layers, viewlets, groklets,
> > additional
> > field indexes and add 'context' to hurry.query)
> Could you elaborate on the additional field indexes you created, and
> you mean by adding context to hurry.query?
Unless they've been added since 0.9, I've cherry-picked a few of the
field indexes from zc.catalog on an as needed basis.
Content is divided by year. So each year is a site with it's own
/++etc++site and it's own Catalog. Currently hurry.query doesn't allow
for context although getUtility does. Here's how I tweaked hurry.query
def __init__(self, (catalog_name, index_name), context=None):
self.catalog_name = catalog_name
self.index_name = index_name
self.context = context
catalog = zapi.getUtility(ICatalog, self.catalog_name,
context=self.context) #using context
index = catalog[self.index_name]
def __init__(self, index_id, text, context=None):
super(Text, self).__init__(index_id, context=context)
self.text = text
index = super(Text, self).getIndex()
index = self.getIndex()
> > The site initially made heavy use of viewlets which were then
> > simplified to groklets.
> It'd be valuable to see your feedback on the viewlets branch. I'd like
> to hear more about how the groklet simplification helped you. We
> decided not to do away with the viewletmanager concept yet as it could
> be made to be quite small with Grok. I noticed that your viewlets
> branch had some code that seemed to break some features of viewlets (I
> forget the details, I think possibly the ability to associate them
> with particular views, but I may be wrong).
For the Zope3 developer, viewlets are a valuable addition to Grok.
Groklets aren't ready for general consumption yet, but it did cut down
my 'viewlet' code by at least a third.
> I hope that you won't be stuck on 0.9 forever and that you will be
> able to move to Grok 1.0+ so we share a common base for these features.
yes, hopefully before Grok 2.0 anyway :)
>> The site's not finished yet, and our business requirements are
>> constantly in flux, but thankfully due to Grok's roots in Zope3, it's
>> always being evolved to meet our needs.
>> THANK YOU Grok community for your hard work and know-how in creating
>> a dream web programming environment.
>> Go GROK!
> Thank you very much for all the contributions you've already made for
> Grok, and this feedback - it's valuable feedback and also quite
> pleasant to see Grok worked out for you!
Thanks you, and hopefully I'll be able to open time to contribute
properly in the future ( I don't even remember implementing the viewlets
> Grok-dev mailing list
> Grok-dev at zope.org
More information about the Grok-dev