[Checkins] SVN: zope.app.publisher/branches/wosc-hashed-resources/ moved work from trunk to branch
Wolfgang Schnerring
wosc at wosc.de
Wed Jun 10 04:19:56 EDT 2009
Log message for revision 100785:
moved work from trunk to branch
Changed:
U zope.app.publisher/branches/wosc-hashed-resources/CHANGES.txt
U zope.app.publisher/branches/wosc-hashed-resources/README.txt
U zope.app.publisher/branches/wosc-hashed-resources/setup.py
U zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/configure.zcml
U zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/resource.py
A zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/hashedresources.zcml
U zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/support.py
A zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/test_hashingurl.py
U zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/interfaces/__init__.py
-=-
Modified: zope.app.publisher/branches/wosc-hashed-resources/CHANGES.txt
===================================================================
--- zope.app.publisher/branches/wosc-hashed-resources/CHANGES.txt 2009-06-10 08:18:27 UTC (rev 100784)
+++ zope.app.publisher/branches/wosc-hashed-resources/CHANGES.txt 2009-06-10 08:19:56 UTC (rev 100785)
@@ -2,11 +2,19 @@
Changes
=======
-3.8.2 (unreleased)
+3.9.0 (unreleased)
==================
-- Remove test dependency on ``zope.app.pagetemplate``.
+- To make browsers update their caches of resources immediately when the
+ resource changes, the absolute URLs of resources can now be made to contain a
+ hash of the resource's contents, so it will look like
+ /++noop++12345/@@/myresource instead of /@@/myresource.
+ This feature is deactivated by default, to activate it, use
+ <meta:provides feature="zope.app.publisher.hashed-resources" />
+
+- Removed test dependency on ``zope.app.pagetemplate``.
+
3.8.1 (2009-05-25)
==================
Modified: zope.app.publisher/branches/wosc-hashed-resources/README.txt
===================================================================
--- zope.app.publisher/branches/wosc-hashed-resources/README.txt 2009-06-10 08:18:27 UTC (rev 100784)
+++ zope.app.publisher/branches/wosc-hashed-resources/README.txt 2009-06-10 08:19:56 UTC (rev 100785)
@@ -86,9 +86,9 @@
Rather than putting together the URL to a resource manually, you should use
zope.traversing.browser.interfaces.IAbsoluteURL to get the URL, or for a
-shorthand, call the resource object. This has an additional benefit:
+shorthand, call the resource object. This has two additional benefits.
-If you want to serve resources from a different URL, for example
+Firstly, if you want to serve resources from a different URL, for example
because you want to use a web server specialized in serving static files instead
of the appserver, you can register an IAbsoluteURL adapter for the site under
the name 'resource' that will be used to compute the base URLs for resources.
@@ -98,3 +98,28 @@
URLs: http://static.example.com/myfile and
http://static.example.com/main-images
(XXX what about http://static.example.com/main-images/subdir/sample.jpg?)
+
+The other benefit of using generated URLs is about dealing with browser caches,
+as described in the next section.
+
+Browser Caches
+~~~~~~~~~~~~~~
+
+While we want browsers to cache static resources such as CSS-stylesheets and
+JavaScript files, we also want them *not* to use the cached version if the
+files on the server have been updated. (And we don't want to make end-users
+have to empty their browser cache to get the latest version. Nor explain how
+to do that over the phone every time.)
+
+To make browsers update their caches of resources immediately when the
+resource changes, the absolute URLs of resources can now be made to contain a
+hash of the resource's contents, so it will look like
+/++noop++12345/@@/myresource instead of /@@/myresource.
+
+In developer mode the hash is recomputed each time the resource is asked for
+its URL, while in production mode the hash is computed only once, so remember
+to restart the server after changing resource files (else browsers will still
+see the old URL unchanged and use their outdated cached versions of the files).
+
+This feature is deactivated by default, to activate it, use
+<meta:provides feature="zope.app.publisher.hashed-resources" />
Modified: zope.app.publisher/branches/wosc-hashed-resources/setup.py
===================================================================
--- zope.app.publisher/branches/wosc-hashed-resources/setup.py 2009-06-10 08:18:27 UTC (rev 100784)
+++ zope.app.publisher/branches/wosc-hashed-resources/setup.py 2009-06-10 08:19:56 UTC (rev 100785)
@@ -19,7 +19,7 @@
open('CHANGES.txt').read())
setup(name='zope.app.publisher',
- version = '3.8.2dev',
+ version = '3.9.0dev',
url='http://pypi.python.org/pypi/zope.app.publisher/',
author='Zope Corporation and Contributors',
author_email='zope-dev at zope.org',
@@ -63,7 +63,9 @@
'zope.app.testing',
'zope.app.securitypolicy',
'zope.app.zcmlfiles',
- 'zope.site'],
+ 'zope.site',
+ 'zope.testbrowser',
+ ],
},
zip_safe = False,
Modified: zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/configure.zcml
===================================================================
--- zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/configure.zcml 2009-06-10 08:18:27 UTC (rev 100784)
+++ zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/configure.zcml 2009-06-10 08:19:56 UTC (rev 100785)
@@ -57,10 +57,35 @@
<allow attributes="get __getitem__" />
</class>
-<adapter
+<adapter zcml:condition="not-have zope.app.publisher.hashed-resources"
factory=".resource.AbsoluteURL"
/>
+<adapter zcml:condition="have zope.app.publisher.hashed-resources"
+ factory=".resource.HashingURL"
+ />
+<adapter zcml:condition="have devmode"
+ factory=".resource.ContentsHash"
+ for=".directoryresource.DirectoryResource"
+ />
+<adapter zcml:condition="have devmode"
+ factory=".resource.ContentsHash"
+ for=".fileresource.FileResource"
+ />
+<adapter zcml:condition="not-have devmode"
+ factory=".resource.CachingContentsHash"
+ for=".directoryresource.DirectoryResource"
+ />
+<adapter zcml:condition="not-have devmode"
+ factory=".resource.CachingContentsHash"
+ for=".fileresource.FileResource"
+ />
+
+<adapter
+ factory=".resource.NoOpTraverser"
+ name="noop"
+ />
+
<browser:page
name=""
for="zope.location.interfaces.ISite"
Modified: zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/resource.py
===================================================================
--- zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/resource.py 2009-06-10 08:18:27 UTC (rev 100784)
+++ zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/resource.py 2009-06-10 08:19:56 UTC (rev 100785)
@@ -15,17 +15,21 @@
$Id$
"""
-from zope.app.publisher.interfaces import IResource
+from zope.app.publisher.interfaces import IResource, IResourceContentsHash
from zope.component import adapts
from zope.component import getMultiAdapter
from zope.component import queryMultiAdapter
-from zope.interface import implements, implementsOnly
+from zope.interface import implements, implementsOnly, Interface
from zope.location import Location
from zope.publisher.interfaces.browser import IDefaultBrowserLayer
from zope.site.hooks import getSite
+from zope.traversing.interfaces import ITraversable
from zope.traversing.browser.interfaces import IAbsoluteURL
+import os
+import md5
import zope.traversing.browser.absoluteurl
+
class Resource(Location):
implements(IResource)
@@ -46,11 +50,11 @@
self.context = context
self.request = request
- def __str__(self):
- name = self.context.__name__
- if name.startswith('++resource++'):
- name = name[12:]
+ self.name = self.context.__name__
+ if self.name.startswith('++resource++'):
+ self.name = self.name[12:]
+ def _site_url(self):
site = getSite()
base = queryMultiAdapter((site, self.request), IAbsoluteURL,
name="resource")
@@ -58,6 +62,79 @@
url = str(getMultiAdapter((site, self.request), IAbsoluteURL))
else:
url = str(base)
+ return url
- return "%s/@@/%s" % (url, name)
+ def __str__(self):
+ return "%s/@@/%s" % (self._site_url(), self.name)
+
+class HashingURL(AbsoluteURL):
+ """Inserts a hash of the contents into the resource's URL,
+ so the URL changes whenever the contents change, thereby forcing
+ a browser to update its cache.
+ """
+
+ def __str__(self):
+ hash = str(IResourceContentsHash(self.context))
+ return "%s/++noop++%s/@@/%s" % (self._site_url(), hash, self.name)
+
+
+class ContentsHash(object):
+
+ implements(zope.app.publisher.interfaces.IResourceContentsHash)
+
+ def __init__(self, context):
+ self.context = context
+
+ def __str__(self):
+ path = self.context.context.path
+ if os.path.isdir(path):
+ files = self._list_directory(path)
+ else:
+ files = [path]
+
+ result = md5.new()
+ for file in files:
+ f = open(file, 'rb')
+ data = f.read()
+ f.close()
+ result.update(data)
+ result = result.hexdigest()
+ return result
+
+ def _list_directory(self, path):
+ for root, dirs, files in os.walk(path):
+ for file in files:
+ yield os.path.join(root, file)
+
+
+_contents_hash = {}
+
+class CachingContentsHash(ContentsHash):
+
+ def __str__(self):
+ path = self.context.context.path
+ try:
+ return _contents_hash[path]
+ except KeyError:
+ result = super(CachingContentsHash, self).__str__()
+ _contents_hash[path] = result
+ return result
+
+
+class NoOpTraverser(object):
+ """This traverser simply skips a path element,
+ so /foo/++noop++qux/bar is equivalent to /foo/bar.
+
+ This is useful to generate varying URLs to work around browser caches.
+ """
+
+ adapts(Interface, IDefaultBrowserLayer)
+ implements(ITraversable)
+
+ def __init__(self, context, request):
+ self.context = context
+ self.request = request
+
+ def traverse(self, name, furtherPath):
+ return self.context
Copied: zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/hashedresources.zcml (from rev 100782, zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/hashedresources.zcml)
===================================================================
--- zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/hashedresources.zcml (rev 0)
+++ zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/hashedresources.zcml 2009-06-10 08:19:56 UTC (rev 100785)
@@ -0,0 +1,18 @@
+<configure
+ xmlns="http://namespaces.zope.org/zope"
+ xmlns:browser="http://namespaces.zope.org/browser"
+ xmlns:meta="http://namespaces.zope.org/meta"
+ i18n_domain="zope"
+ package="zope.app.publisher"
+ >
+
+ <meta:provides feature="zope.app.publisher.hashed-resources" />
+
+ <include package="zope.app.publisher" file="ftesting.zcml" />
+
+ <!-- example resource for testing -->
+ <browser:resourceDirectory
+ directory="browser/tests/testfiles"
+ name="myresource"
+ />
+</configure>
\ No newline at end of file
Modified: zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/support.py
===================================================================
--- zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/support.py 2009-06-10 08:18:27 UTC (rev 100784)
+++ zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/support.py 2009-06-10 08:19:56 UTC (rev 100785)
@@ -45,6 +45,8 @@
def tearDown(self):
setSite()
+ zope.component.getSiteManager().unregisterAdapter(
+ zope.app.publisher.browser.resource.AbsoluteURL)
super(SiteHandler, self).tearDown()
Copied: zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/test_hashingurl.py (from rev 100782, zope.app.publisher/trunk/src/zope/app/publisher/browser/tests/test_hashingurl.py)
===================================================================
--- zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/test_hashingurl.py (rev 0)
+++ zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/browser/tests/test_hashingurl.py 2009-06-10 08:19:56 UTC (rev 100785)
@@ -0,0 +1,144 @@
+##############################################################################
+#
+# Copyright (c) 2001, 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.1 (ZPL). A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Test for hashed resource-URLs
+
+$Id: test_directoryresource.py 95447 2009-01-29 16:28:18Z wosc $
+"""
+import os
+import re
+import tempfile
+import unittest
+import zope.app.publisher.browser.directoryresource
+import zope.app.publisher.browser.tests
+import zope.app.publisher.testing
+import zope.app.testing.functional
+import zope.app.testing.placelesssetup
+import zope.publisher.browser
+import zope.security.checker
+import zope.site.hooks
+import zope.testbrowser.testing
+
+fixture = os.path.join(
+ os.path.dirname(zope.app.publisher.browser.tests.__file__), 'testfiles')
+
+checker = zope.security.checker.NamesChecker(
+ ('get', '__getitem__', 'request', 'publishTraverse')
+ )
+
+HashedResourcesLayer = zope.app.testing.functional.ZCMLLayer(
+ os.path.join(os.path.split(__file__)[0], 'hashedresources.zcml'),
+ __name__, 'HashedResourcesLayer', allow_teardown=True)
+
+
+class HashingURLTest(zope.app.testing.functional.FunctionalTestCase):
+
+ layer = HashedResourcesLayer
+
+ def assertMatches(self, regex, text):
+ self.assert_(re.match(regex, text), "/%s/ did not match '%s'" % (
+ regex, text))
+
+ def setUp(self):
+ super(HashingURLTest, self).setUp()
+ self.site = zope.site.hooks.getSite()
+
+ self.tmpdir = tempfile.mkdtemp()
+ open(os.path.join(self.tmpdir, 'example.txt'), 'w').write('')
+ self.dirname = os.path.basename(self.tmpdir)
+
+ self.request = zope.publisher.browser.TestRequest()
+ self.request._vh_root = self.site
+ self.directory = zope.app.publisher.browser.directoryresource.DirectoryResourceFactory(
+ self.tmpdir, checker, self.dirname)(self.request)
+ self.directory.__parent__ = self.site
+
+ def _hash(self, text):
+ return re.match('http://127.0.0.1/\+\+noop\+\+([^/]*)/.*', text).group(1)
+
+ def test_directory_url_should_contain_hash(self):
+ self.assertMatches(
+ 'http://127.0.0.1/\+\+noop\+\+[^/]*/@@/%s' % self.dirname, self.directory())
+
+ def test_file_url_should_contain_hash(self):
+ file = zope.app.publisher.browser.fileresource.FileResourceFactory(
+ os.path.join(fixture, 'test.txt'), checker, 'test.txt')(self.request)
+ self.assertMatches(
+ 'http://127.0.0.1/\+\+noop\+\+[^/]*/@@/test.txt', file())
+
+ def test_different_files_hashes_should_differ(self):
+ file1 = zope.app.publisher.browser.fileresource.FileResourceFactory(
+ os.path.join(fixture, 'test.txt'), checker, 'test.txt')(self.request)
+ file2 = zope.app.publisher.browser.fileresource.FileResourceFactory(
+ os.path.join(fixture, 'test.pt'), checker, 'test.txt')(self.request)
+ self.assertNotEqual(self._hash(file1()), self._hash(file2()))
+
+ def test_directory_contents_changed_hash_should_change(self):
+ before = self._hash(self.directory())
+ open(os.path.join(self.tmpdir, 'example.txt'), 'w').write('foo')
+ after = self._hash(self.directory())
+ self.assertNotEqual(before, after)
+
+
+class DeveloperModeTest(HashingURLTest):
+
+ def test_production_mode_hash_should_not_change(self):
+ zope.component.provideAdapter(
+ zope.app.publisher.browser.resource.CachingContentsHash,
+ (zope.app.publisher.browser.directoryresource.DirectoryResource,))
+
+ before = self._hash(self.directory())
+ open(os.path.join(self.tmpdir, 'example.txt'), 'w').write('foo')
+ after = self._hash(self.directory())
+ self.assertEqual(before, after)
+
+
+class BrowserTest(zope.app.testing.functional.FunctionalTestCase):
+
+ layer = HashedResourcesLayer
+
+ def setUp(self):
+ super(BrowserTest, self).setUp()
+ self.browser = zope.testbrowser.testing.Browser()
+ self.directory = zope.component.getAdapter(
+ zope.publisher.browser.TestRequest(), name='myresource')
+
+ def test_traverse_atat_by_name(self):
+ self.browser.open('http://localhost/@@/myresource/test.txt')
+ self.assertEqual('test\ndata\n', self.browser.contents)
+
+ def test_traverse_atat_by_hash(self):
+ hash = str(
+ zope.app.publisher.interfaces.IResourceContentsHash(self.directory))
+ self.browser.open(
+ 'http://localhost/++noop++%s/@@/myresource/test.txt' % hash)
+ self.assertEqual('test\ndata\n', self.browser.contents)
+
+ def test_traverse_resource_by_name(self):
+ self.browser.open('http://localhost/++resource++myresource/test.txt')
+ self.assertEqual('test\ndata\n', self.browser.contents)
+
+ def test_traverse_resource_by_hash(self):
+ hash = str(
+ zope.app.publisher.interfaces.IResourceContentsHash(self.directory))
+ self.browser.open(
+ 'http://localhost/++noop++%s/++resource++myresource/test.txt' % hash)
+ self.assertEqual('test\ndata\n', self.browser.contents)
+
+
+def test_suite():
+ suite = unittest.TestSuite()
+ suite.addTest(unittest.makeSuite(HashingURLTest))
+ suite.addTest(unittest.makeSuite(DeveloperModeTest))
+ suite.addTest(unittest.makeSuite(BrowserTest))
+ return suite
Modified: zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/interfaces/__init__.py
===================================================================
--- zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/interfaces/__init__.py 2009-06-10 08:18:27 UTC (rev 100784)
+++ zope.app.publisher/branches/wosc-hashed-resources/src/zope/app/publisher/interfaces/__init__.py 2009-06-10 08:19:56 UTC (rev 100785)
@@ -24,3 +24,9 @@
def __call__():
"""return the absolute URL of this resource."""
+
+
+class IResourceContentsHash(Interface):
+
+ def __str__():
+ """return a hash of the contents of the resource"""
More information about the Checkins
mailing list