[Checkins] SVN: zope.app.publisher/trunk/ To make browsers update their caches of resources immediately when the
Wolfgang Schnerring
wosc at wosc.de
Tue Jun 9 07:55:23 EDT 2009
Log message for revision 100756:
To make browsers update their caches of resources immediately when the
resource changes, the absolute URLs of resources can now be made to contain a
hash of the resource's contents, so it will look like
/++noop++12345/@@/myresource instead of /@@/myresource.
- Implemented an AbsoluteURL adapter that computes a hash of the resource's contents and inserts that into the URL.
- Content hashes will be cached in memory, except when in developer mode
- Introduced a ++noop++ traverser that simply throws away the path segment
- Wrote a bit of documentation about resources
Changed:
U zope.app.publisher/trunk/CHANGES.txt
U zope.app.publisher/trunk/README.txt
U zope.app.publisher/trunk/setup.py
U zope.app.publisher/trunk/src/zope/app/publisher/browser/configure.zcml
U zope.app.publisher/trunk/src/zope/app/publisher/browser/resource.py
A zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/hashedresources.zcml
U zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/support.py
A zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/test_hashingurl.py
U zope.app.publisher/trunk/src/zope/app/publisher/interfaces/__init__.py
-=-
Modified: zope.app.publisher/trunk/CHANGES.txt
===================================================================
--- zope.app.publisher/trunk/CHANGES.txt 2009-06-09 09:34:46 UTC (rev 100755)
+++ zope.app.publisher/trunk/CHANGES.txt 2009-06-09 11:55:23 UTC (rev 100756)
@@ -2,11 +2,19 @@
Changes
=======
-3.8.2 (unreleased)
+3.9.0 (unreleased)
==================
-- Remove test dependency on ``zope.app.pagetemplate``.
+- To make browsers update their caches of resources immediately when the
+ resource changes, the absolute URLs of resources can now be made to contain a
+ hash of the resource's contents, so it will look like
+ /++noop++12345/@@/myresource instead of /@@/myresource.
+ This feature is deactivated by default, to activate it, use
+ <meta:provides feature="zope.app.publisher.hashed-resources" />
+
+- Removed test dependency on ``zope.app.pagetemplate``.
+
3.8.1 (2009-05-25)
==================
Modified: zope.app.publisher/trunk/README.txt
===================================================================
--- zope.app.publisher/trunk/README.txt 2009-06-09 09:34:46 UTC (rev 100755)
+++ zope.app.publisher/trunk/README.txt 2009-06-09 11:55:23 UTC (rev 100756)
@@ -2,8 +2,8 @@
Overview
========
-*This package is at present not reusable without depending on a large
-chunk of the Zope Toolkit and its assumptions. It is maintained by the*
+*This package is at present not reusable without depending on a large
+chunk of the Zope Toolkit and its assumptions. It is maintained by the*
`Zope Toolkit project <http://docs.zope.org/zopetoolkit/>`_.
``zope.publisher`` is a general purpose object publishing framework
@@ -34,8 +34,92 @@
* browser:resourceDirectory
-* browser:defeaultSkin
+* browser:defaultSkin
* browser:icon
* xmlrpc:view
+
+
+Views and Browser pages
+=======================
+
+XXX writeme
+
+
+Resources
+=========
+
+Resources are static files and directories that are served to the browser
+directly from the filesystem. The most common example are images, CSS style
+sheets, or JavaScript files.
+
+Resources are be registered under a symbolic name and can later be referred to
+by that name, so their usage is independent from their physical location.
+
+You can register a single file with the `<browser:resource>` directive, and a
+whole directory with the `<browser:resourceDirectory>` directive, for example
+
+ <browser:resource
+ directory="/path/to/static.file"
+ name="myfile"
+ />
+
+ <browser:resourceDirectory
+ directory="/path/to/images"
+ name="main-images"
+ />
+
+This causes a named adapter to be registered that adapts the request to
+zope.interface.Interface (XXX why do we not use an explicit interface?),
+so to later retrieve a resource, use
+`zope.component.getAdapter(request, name='myfile')`.
+
+There are two ways to traverse to a resource,
+1. with the 'empty' view on a site, e. g. `http://localhost/@@/myfile`
+ (This is declared by zope.app.publisher.browser)
+2. with the `++resource++` namespace, e. g. `http://localhost/++resource++myfile`
+ (This is declared by zope.traversing.namespace)
+
+In case of resource-directories traversal simply continues through its contents,
+e. g. `http://localhost/@@/main-images/subdir/sample.jpg`
+
+Rather than putting together the URL to a resource manually, you should use
+zope.traversing.browser.interfaces.IAbsoluteURL to get the URL, or for a
+shorthand, call the resource object. This has two additional benefits.
+
+Firstly, if you want to serve resources from a different URL, for example
+because you want to use a web server specialized in serving static files instead
+of the appserver, you can register an IAbsoluteURL adapter for the site under
+the name 'resource' that will be used to compute the base URLs for resources.
+
+For example, if you register 'http://static.example.com/' as the base 'resource'
+URL, the resources from the above example would yield the following absolute
+URLs: http://static.example.com/myfile and
+http://static.example.com/main-images
+(XXX what about http://static.example.com/main-images/subdir/sample.jpg?)
+
+The other benefit of using generated URLs is about dealing with browser caches,
+as described in the next section.
+
+Browser Caches
+~~~~~~~~~~~~~~
+
+While we want browsers to cache static resources such as CSS-stylesheets and
+JavaScript files, we also want them *not* to use the cached version if the
+files on the server have been updated. (And we don't want to make end-users
+have to empty their browser cache to get the latest version. Nor explain how
+to do that over the phone every time.)
+
+To make browsers update their caches of resources immediately when the
+resource changes, the absolute URLs of resources can now be made to contain a
+hash of the resource's contents, so it will look like
+/++noop++12345/@@/myresource instead of /@@/myresource.
+
+In developer mode the hash is recomputed each time the resource is asked for
+its URL, while in production mode the hash is computed only once, so remember
+to restart the server after changing resource files (else browsers will still
+see the old URL unchanged and use their outdated cached versions of the files).
+
+This feature is deactivated by default, to activate it, use
+<meta:provides feature="zope.app.publisher.hashed-resources" />
Modified: zope.app.publisher/trunk/setup.py
===================================================================
--- zope.app.publisher/trunk/setup.py 2009-06-09 09:34:46 UTC (rev 100755)
+++ zope.app.publisher/trunk/setup.py 2009-06-09 11:55:23 UTC (rev 100756)
@@ -19,7 +19,7 @@
open('CHANGES.txt').read())
setup(name='zope.app.publisher',
- version = '3.8.2dev',
+ version = '3.9.0dev',
url='http://pypi.python.org/pypi/zope.app.publisher/',
author='Zope Corporation and Contributors',
author_email='zope-dev at zope.org',
@@ -63,7 +63,9 @@
'zope.app.testing',
'zope.app.securitypolicy',
'zope.app.zcmlfiles',
- 'zope.site'],
+ 'zope.site',
+ 'zope.testbrowser',
+ ],
},
zip_safe = False,
Modified: zope.app.publisher/trunk/src/zope/app/publisher/browser/configure.zcml
===================================================================
--- zope.app.publisher/trunk/src/zope/app/publisher/browser/configure.zcml 2009-06-09 09:34:46 UTC (rev 100755)
+++ zope.app.publisher/trunk/src/zope/app/publisher/browser/configure.zcml 2009-06-09 11:55:23 UTC (rev 100756)
@@ -57,10 +57,35 @@
<allow attributes="get __getitem__" />
</class>
-<adapter
+<adapter zcml:condition="not-have zope.app.publisher.hashed-resources"
factory=".resource.AbsoluteURL"
/>
+<adapter zcml:condition="have zope.app.publisher.hashed-resources"
+ factory=".resource.HashingURL"
+ />
+<adapter zcml:condition="have devmode"
+ factory=".resource.ContentsHash"
+ for=".directoryresource.DirectoryResource"
+ />
+<adapter zcml:condition="have devmode"
+ factory=".resource.ContentsHash"
+ for=".fileresource.FileResource"
+ />
+<adapter zcml:condition="not-have devmode"
+ factory=".resource.CachingContentsHash"
+ for=".directoryresource.DirectoryResource"
+ />
+<adapter zcml:condition="not-have devmode"
+ factory=".resource.CachingContentsHash"
+ for=".fileresource.FileResource"
+ />
+
+<adapter
+ factory=".resource.NoOpTraverser"
+ name="noop"
+ />
+
<browser:page
name=""
for="zope.location.interfaces.ISite"
Modified: zope.app.publisher/trunk/src/zope/app/publisher/browser/resource.py
===================================================================
--- zope.app.publisher/trunk/src/zope/app/publisher/browser/resource.py 2009-06-09 09:34:46 UTC (rev 100755)
+++ zope.app.publisher/trunk/src/zope/app/publisher/browser/resource.py 2009-06-09 11:55:23 UTC (rev 100756)
@@ -15,17 +15,21 @@
$Id$
"""
-from zope.app.publisher.interfaces import IResource
+from zope.app.publisher.interfaces import IResource, IResourceContentsHash
from zope.component import adapts
from zope.component import getMultiAdapter
from zope.component import queryMultiAdapter
-from zope.interface import implements, implementsOnly
+from zope.interface import implements, implementsOnly, Interface
from zope.location import Location
from zope.publisher.interfaces.browser import IDefaultBrowserLayer
from zope.site.hooks import getSite
+from zope.traversing.interfaces import ITraversable
from zope.traversing.browser.interfaces import IAbsoluteURL
+import os
+import md5
import zope.traversing.browser.absoluteurl
+
class Resource(Location):
implements(IResource)
@@ -46,11 +50,11 @@
self.context = context
self.request = request
- def __str__(self):
- name = self.context.__name__
- if name.startswith('++resource++'):
- name = name[12:]
+ self.name = self.context.__name__
+ if self.name.startswith('++resource++'):
+ self.name = self.name[12:]
+ def _site_url(self):
site = getSite()
base = queryMultiAdapter((site, self.request), IAbsoluteURL,
name="resource")
@@ -58,6 +62,79 @@
url = str(getMultiAdapter((site, self.request), IAbsoluteURL))
else:
url = str(base)
+ return url
- return "%s/@@/%s" % (url, name)
+ def __str__(self):
+ return "%s/@@/%s" % (self._site_url(), self.name)
+
+class HashingURL(AbsoluteURL):
+ """Inserts a hash of the contents into the resource's URL,
+ so the URL changes whenever the contents change, thereby forcing
+ a browser to update its cache.
+ """
+
+ def __str__(self):
+ hash = str(IResourceContentsHash(self.context))
+ return "%s/++noop++%s/@@/%s" % (self._site_url(), hash, self.name)
+
+
+class ContentsHash(object):
+
+ implements(zope.app.publisher.interfaces.IResourceContentsHash)
+
+ def __init__(self, context):
+ self.context = context
+
+ def __str__(self):
+ path = self.context.context.path
+ if os.path.isdir(path):
+ files = self._list_directory(path)
+ else:
+ files = [path]
+
+ result = md5.new()
+ for file in files:
+ f = open(file, 'rb')
+ data = f.read()
+ f.close()
+ result.update(data)
+ result = result.hexdigest()
+ return result
+
+ def _list_directory(self, path):
+ for root, dirs, files in os.walk(path):
+ for file in files:
+ yield os.path.join(root, file)
+
+
+_contents_hash = {}
+
+class CachingContentsHash(ContentsHash):
+
+ def __str__(self):
+ path = self.context.context.path
+ try:
+ return _contents_hash[path]
+ except KeyError:
+ result = super(CachingContentsHash, self).__str__()
+ _contents_hash[path] = result
+ return result
+
+
+class NoOpTraverser(object):
+ """This traverser simply skips a path element,
+ so /foo/++noop++qux/bar is equivalent to /foo/bar.
+
+ This is useful to generate varying URLs to work around browser caches.
+ """
+
+ adapts(Interface, IDefaultBrowserLayer)
+ implements(ITraversable)
+
+ def __init__(self, context, request):
+ self.context = context
+ self.request = request
+
+ def traverse(self, name, furtherPath):
+ return self.context
Added: zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/hashedresources.zcml
===================================================================
--- zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/hashedresources.zcml (rev 0)
+++ zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/hashedresources.zcml 2009-06-09 11:55:23 UTC (rev 100756)
@@ -0,0 +1,18 @@
+<configure
+ xmlns="http://namespaces.zope.org/zope"
+ xmlns:browser="http://namespaces.zope.org/browser"
+ xmlns:meta="http://namespaces.zope.org/meta"
+ i18n_domain="zope"
+ package="zope.app.publisher"
+ >
+
+ <meta:provides feature="zope.app.publisher.hashed-resources" />
+
+ <include package="zope.app.publisher" file="ftesting.zcml" />
+
+ <!-- example resource for testing -->
+ <browser:resourceDirectory
+ directory="browser/tests/testfiles"
+ name="myresource"
+ />
+</configure>
\ No newline at end of file
Modified: zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/support.py
===================================================================
--- zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/support.py 2009-06-09 09:34:46 UTC (rev 100755)
+++ zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/support.py 2009-06-09 11:55:23 UTC (rev 100756)
@@ -45,6 +45,8 @@
def tearDown(self):
setSite()
+ zope.component.getSiteManager().unregisterAdapter(
+ zope.app.publisher.browser.resource.AbsoluteURL)
super(SiteHandler, self).tearDown()
Added: zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/test_hashingurl.py
===================================================================
--- zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/test_hashingurl.py (rev 0)
+++ zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/test_hashingurl.py 2009-06-09 11:55:23 UTC (rev 100756)
@@ -0,0 +1,144 @@
+##############################################################################
+#
+# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Test for hashed resource-URLs
+
+$Id: test_directoryresource.py 95447 2009-01-29 16:28:18Z wosc $
+"""
+import os
+import re
+import tempfile
+import unittest
+import zope.app.publisher.browser.directoryresource
+import zope.app.publisher.browser.tests
+import zope.app.publisher.testing
+import zope.app.testing.functional
+import zope.app.testing.placelesssetup
+import zope.publisher.browser
+import zope.security.checker
+import zope.site.hooks
+import zope.testbrowser.testing
+
+fixture = os.path.join(
+ os.path.dirname(zope.app.publisher.browser.tests.__file__), 'testfiles')
+
+checker = zope.security.checker.NamesChecker(
+ ('get', '__getitem__', 'request', 'publishTraverse')
+ )
+
+HashedResourcesLayer = zope.app.testing.functional.ZCMLLayer(
+ os.path.join(os.path.split(__file__)[0], 'hashedresources.zcml'),
+ __name__, 'HashedResourcesLayer', allow_teardown=True)
+
+
+class HashingURLTest(zope.app.testing.functional.FunctionalTestCase):
+
+ layer = HashedResourcesLayer
+
+ def assertMatches(self, regex, text):
+ self.assert_(re.match(regex, text), "/%s/ did not match '%s'" % (
+ regex, text))
+
+ def setUp(self):
+ super(HashingURLTest, self).setUp()
+ self.site = zope.site.hooks.getSite()
+
+ self.tmpdir = tempfile.mkdtemp()
+ open(os.path.join(self.tmpdir, 'example.txt'), 'w').write('')
+ self.dirname = os.path.basename(self.tmpdir)
+
+ self.request = zope.publisher.browser.TestRequest()
+ self.request._vh_root = self.site
+ self.directory = zope.app.publisher.browser.directoryresource.DirectoryResourceFactory(
+ self.tmpdir, checker, self.dirname)(self.request)
+ self.directory.__parent__ = self.site
+
+ def _hash(self, text):
+ return re.match('http://127.0.0.1/\+\+noop\+\+([^/]*)/.*', text).group(1)
+
+ def test_directory_url_should_contain_hash(self):
+ self.assertMatches(
+ 'http://127.0.0.1/\+\+noop\+\+[^/]*/@@/%s' % self.dirname, self.directory())
+
+ def test_file_url_should_contain_hash(self):
+ file = zope.app.publisher.browser.fileresource.FileResourceFactory(
+ os.path.join(fixture, 'test.txt'), checker, 'test.txt')(self.request)
+ self.assertMatches(
+ 'http://127.0.0.1/\+\+noop\+\+[^/]*/@@/test.txt', file())
+
+ def test_different_files_hashes_should_differ(self):
+ file1 = zope.app.publisher.browser.fileresource.FileResourceFactory(
+ os.path.join(fixture, 'test.txt'), checker, 'test.txt')(self.request)
+ file2 = zope.app.publisher.browser.fileresource.FileResourceFactory(
+ os.path.join(fixture, 'test.pt'), checker, 'test.txt')(self.request)
+ self.assertNotEqual(self._hash(file1()), self._hash(file2()))
+
+ def test_directory_contents_changed_hash_should_change(self):
+ before = self._hash(self.directory())
+ open(os.path.join(self.tmpdir, 'example.txt'), 'w').write('foo')
+ after = self._hash(self.directory())
+ self.assertNotEqual(before, after)
+
+
+class DeveloperModeTest(HashingURLTest):
+
+ def test_production_mode_hash_should_not_change(self):
+ zope.component.provideAdapter(
+ zope.app.publisher.browser.resource.CachingContentsHash,
+ (zope.app.publisher.browser.directoryresource.DirectoryResource,))
+
+ before = self._hash(self.directory())
+ open(os.path.join(self.tmpdir, 'example.txt'), 'w').write('foo')
+ after = self._hash(self.directory())
+ self.assertEqual(before, after)
+
+
+class BrowserTest(zope.app.testing.functional.FunctionalTestCase):
+
+ layer = HashedResourcesLayer
+
+ def setUp(self):
+ super(BrowserTest, self).setUp()
+ self.browser = zope.testbrowser.testing.Browser()
+ self.directory = zope.component.getAdapter(
+ zope.publisher.browser.TestRequest(), name='myresource')
+
+ def test_traverse_atat_by_name(self):
+ self.browser.open('http://localhost/@@/myresource/test.txt')
+ self.assertEqual('test\ndata\n', self.browser.contents)
+
+ def test_traverse_atat_by_hash(self):
+ hash = str(
+ zope.app.publisher.interfaces.IResourceContentsHash(self.directory))
+ self.browser.open(
+ 'http://localhost/++noop++%s/@@/myresource/test.txt' % hash)
+ self.assertEqual('test\ndata\n', self.browser.contents)
+
+ def test_traverse_resource_by_name(self):
+ self.browser.open('http://localhost/++resource++myresource/test.txt')
+ self.assertEqual('test\ndata\n', self.browser.contents)
+
+ def test_traverse_resource_by_hash(self):
+ hash = str(
+ zope.app.publisher.interfaces.IResourceContentsHash(self.directory))
+ self.browser.open(
+ 'http://localhost/++noop++%s/++resource++myresource/test.txt' % hash)
+ self.assertEqual('test\ndata\n', self.browser.contents)
+
+
+def test_suite():
+ suite = unittest.TestSuite()
+ suite.addTest(unittest.makeSuite(HashingURLTest))
+ suite.addTest(unittest.makeSuite(DeveloperModeTest))
+ suite.addTest(unittest.makeSuite(BrowserTest))
+ return suite
Modified: zope.app.publisher/trunk/src/zope/app/publisher/interfaces/__init__.py
===================================================================
--- zope.app.publisher/trunk/src/zope/app/publisher/interfaces/__init__.py 2009-06-09 09:34:46 UTC (rev 100755)
+++ zope.app.publisher/trunk/src/zope/app/publisher/interfaces/__init__.py 2009-06-09 11:55:23 UTC (rev 100756)
@@ -19,5 +19,14 @@
class IResource(Interface):
-
+
request = Attribute('Request object that is requesting the resource')
+
+ def __call__():
+ """return the absolute URL of this resource."""
+
+
+class IResourceContentsHash(Interface):
+
+ def __str__():
+ """return a hash of the contents of the resource"""
More information about the Checkins
mailing list