[ZWeb] Re: zope.org - serious caching issues

[Andrew Sawyers]

On Mon, 2007-05-21 at 10:09 -0400, Jim Fulton wrote:
> I'm adding zope-web to the CC list.
> I wish you hadn't done that yet.  If we keep changing things. it will  
> be hard to figure this out.
> 
> It would be helpful to show the results of, say wget -S, as in:
> 
> jim at ds9:~/tmp$ wget -S http://www.zope.org/news.rss
> --09:58:05--  http://www.zope.org/news.rss
>             => `news.rss'
> Resolving www.zope.org... 63.240.213.171
> Connecting to www.zope.org|63.240.213.171|:80... connected.
> HTTP request sent, awaiting response...
>    HTTP/1.0 200 OK
>    Server: Zope/(unreleased version, python 2.2.3, linux2) ZServer/1.1b1
>    Date: Mon, 21 May 2007 13:57:28 GMT
>    Content-Length: 4011
>    Content-Type: text/xml
>    Age: 4
>    X-Cache: HIT from parent-ng2.zmh.zope.net
>    X-Cache: MISS from cache2.zmh.zope.net
>    Connection: close
> Length: 4,011 (3.9K) [text/xml]
> 
> 100%[====================================>] 4,011         --.--K/s
> 
> 09:58:05 (365.80 KB/s) - `news.rss' saved [4011/4011]

> 
> jim at ds9:~/tmp$ wget --user jim --password xxxxxx -S http:// 
> www.zope.org/news.rss
> --10:00:45--  http://www.zope.org/news.rss
>             => `news.rss.1'
> Resolving www.zope.org... 63.240.213.171
> Connecting to www.zope.org|63.240.213.171|:80... connected.
> HTTP request sent, awaiting response...
>    HTTP/1.0 200 OK
>    Server: Zope/(unreleased version, python 2.2.3, linux2) ZServer/1.1b1
>    Date: Mon, 21 May 2007 13:45:20 GMT
>    Content-Length: 4011
>    Content-Type: text/xml
>    X-Cache: HIT from parent-ng2.zmh.zope.net
>    Age: 892
>    X-Cache: HIT from cache4.zmh.zope.net
>    Connection: close
> Length: 4,011 (3.9K) [text/xml]
> 
> 100%[====================================>] 4,011         --.--K/s
> 
> 10:00:45 (1.85 MB/s) - `news.rss.1' saved [4011/4011]
It looks to me like the cache tier has been changed; IIRC zope.org was
not in the cache tier I setup for Managed Hosting....and didn't have (at
least) 4 cache servers in it's request flow.
> 
> Note that the second request is authenticated (except with a  
> different password :)
That won't make any difference...they both resulted in a cache hit.  The
first time, you hit a front side cache server that didn't have it
cached: cache2, the second time, you hit a cache server that did have it
cached:  cache4.
> 
> ...
> 
> >> The objects do not display any caching policy in ZMI, but the cache
> >> manager still shows the enties in different variations.
What cache manager?
> 
> Possibly because it doesn't know about the change.
> 
> 
> > It looks like in issue in Zope.
> 
> How so?  If you look at the wget output above, there don't seem to be  
> any cache headers set.  So, data would not be cached unless there is  
> an overriding policy in squid.
If the cache tier was changed and relying on the (old) default settings,
then it's going to cache for a certain period of time.
> 
> > If you see both te child and the parent MISS,
> > then what you're getting is coming from the app server.
Yup.
> 
> I'm getting a hit from the parent.  Also note that both hits have me  
> results for which the most recent entry is from March 29.  If I bust  
> the cache with a query string, the most recent entry is for May 15.
> 
> 
> > That would also
> > explain differences based on roles. There is nothing in squid that
> > distinguishes if a user is authenticated, anonymous or manager.
I don't follow this, what explains the difference; there should be no
difference based on roles.
> 
> I *think* Andrew Sawyers did something to arrange that non-anonymous  
> users get non-cached results.  This doesn't seem to be working any  
> more. This is bad. I'm hoping that however got this working properly  
> at some point can tell us what they did. :)
It's been a long time, but IIRC we got rid of all caching for zope.org
where a cache header wasn't explicitly being set.
> 
> Jim
Was the cache servers changed around?  IIRC the zope.org tier only had 2
measly caches....and if they changed, I bet the rules where not changed
along with the cache servers.

Andrew