[Zope] Zope + htdig indexing problem
Wed, 17 Jul 2002 15:09:34 -0600
The best bet is to stop using relative URL's in zope,
but if that causes a lot of pain, here's a couple of other ideas:
Try putting a rule in your robots.txt file such as:
Or, try using the max_hop_count parameter in the htdig.conf file. You'd
still get some repeats, but at least it would stop at some point. This
is only reliable for complete indexes rather than updates.
Dieter Maurer wrote:
>Tiffany Webb writes:
> > We are having a problem with htdig indexing Zope documents with
> > multiple directory listings from htdig -i -vvv:
>Looks like a non-trivial relative URL reference. A relative URL
>reference is non-trivial when it contains a "/" which is not preceeded
>Due to acquisition, Zope resolves such URL references quite well.
>But, when you have a reference cycle containing one (or more)
>non-trivial URL references, then the URLs get longer and longer
>for each round through the circle. Humans finally stop
>turning around the circle, but spiders may be stupid...
>Zope maillist - Zope@zope.org
>** No cross posts or HTML encoding! **
>(Related lists -
> http://lists.zope.org/mailman/listinfo/zope-dev )