[Checkins] SVN: hurry.resource/trunk/src/hurry/resource/ Improved "rollup" story, allowing multiple potential rollups per
Martijn Faassen
faassen at infrae.com
Mon Sep 22 15:49:03 EDT 2008
Log message for revision 91367:
Improved "rollup" story, allowing multiple potential rollups per
resource, and picking the one that rolls up most in that case.
Changed:
U hurry.resource/trunk/src/hurry/resource/README.txt
U hurry.resource/trunk/src/hurry/resource/core.py
U hurry.resource/trunk/src/hurry/resource/interfaces.py
-=-
Modified: hurry.resource/trunk/src/hurry/resource/README.txt
===================================================================
--- hurry.resource/trunk/src/hurry/resource/README.txt 2008-09-22 19:41:09 UTC (rev 91366)
+++ hurry.resource/trunk/src/hurry/resource/README.txt 2008-09-22 19:49:02 UTC (rev 91367)
@@ -295,16 +295,17 @@
>>> needed.resources(mode='debug')
[<Resource 'a2-debug.js' in library 'foo'>]
-Consolidation
--------------
+"Rollups"
+---------
For performance reasons it's often useful to consolidate multiple
-resources into a single, larger resource. Multiple javascript files
-could for instance be offered in a single, larger one. These
-consolidations can be specified when specifying the resource::
+resources into a single, larger resource, a so-called
+"rollup". Multiple javascript files could for instance be offered in a
+single, larger one. These consolidations can be specified when
+specifying the resource::
- >>> b1 = ResourceSpec(foo, 'b1.js', part_of='giant.js')
- >>> b2 = ResourceSpec(foo, 'b2.js', part_of='giant.js')
+ >>> b1 = ResourceSpec(foo, 'b1.js', part_of=['giant.js'])
+ >>> b2 = ResourceSpec(foo, 'b2.js', part_of=['giant.js'])
If we find multiple resources that are also part of a consolidation, the
system automatically collapses them::
@@ -326,9 +327,10 @@
>>> needed.resources()
[<Resource 'b1.js' in library 'foo'>]
-``part_of`` can also be expressed as a fully specified ``ResourceSpec``::
+``part_of`` can also be expressed as a list of fully specified
+``ResourceSpec``::
- >>> b3 = ResourceSpec(foo, 'b3.js', part_of=ResourceSpec(foo, 'giant.js'))
+ >>> b3 = ResourceSpec(foo, 'b3.js', part_of=[ResourceSpec(foo, 'giant.js')])
>>> inclusion3 = Inclusion([b1, b2, b3])
>>> needed = NeededInclusions()
>>> needed.need(inclusion3)
@@ -338,12 +340,12 @@
Consolidation also can work with modes::
>>> b4 = ResourceSpec(foo, 'b4.js',
- ... part_of='giant.js',
- ... debug=ResourceSpec(foo, 'b4-debug.js', part_of='giant-debug.js'))
+ ... part_of=['giant.js'],
+ ... debug=ResourceSpec(foo, 'b4-debug.js', part_of=['giant-debug.js']))
>>> b5 = ResourceSpec(foo, 'b5.js',
- ... part_of='giant.js',
- ... debug=ResourceSpec(foo, 'b5-debug.js', part_of='giant-debug.js'))
+ ... part_of=['giant.js'],
+ ... debug=ResourceSpec(foo, 'b5-debug.js', part_of=['giant-debug.js']))
>>> inclusion4 = Inclusion([b4, b5])
>>> needed = NeededInclusions()
@@ -353,6 +355,21 @@
>>> needed.resources(mode='debug')
[<Resource 'giant-debug.js' in library 'foo'>]
+A resource can be part of multiple rollups. In this case the rollup that
+rolls up the most resources is used::
+
+ >>> b6 = ResourceSpec(foo, 'b6.js',
+ ... part_of=['giant.js', 'even_bigger.js'])
+ >>> b7 = ResourceSpec(foo, 'b7.js',
+ ... part_of=['giant.js', 'even_bigger.js'])
+ >>> b8 = ResourceSpec(foo, 'b8.js',
+ ... part_of=['even_bigger.js'])
+ >>> inclusion5 = Inclusion([b6, b7, b8])
+ >>> needed = NeededInclusions()
+ >>> needed.need(inclusion5)
+ >>> needed.resources()
+ [<Resource 'even_bigger.js' in library 'foo'>]
+
Rendering resources
-------------------
Modified: hurry.resource/trunk/src/hurry/resource/core.py
===================================================================
--- hurry.resource/trunk/src/hurry/resource/core.py 2008-09-22 19:41:09 UTC (rev 91366)
+++ hurry.resource/trunk/src/hurry/resource/core.py 2008-09-22 19:49:02 UTC (rev 91367)
@@ -28,16 +28,27 @@
relpath - the relative path from the root of the library indicating
the actual resource
part_of - optionally, a resource is also part of a larger
- consolidated resource. This can be a path to the
- larger resource in the same library, or a fully
- specified ResourceSpec.
+ consolidated resource. This can be a list of paths to the
+ larger resource in the same library. A list entry can also
+ be a fully specified ResourceSpec.
key word arguments - different paths that represent the same
resource in different modes (debug, minified, etc),
or alternatively a fully specified ResourceSpec.
"""
self.library = library
self.relpath = relpath
- self.part_of = part_of
+
+ assert not isinstance(part_of, basestring)
+ part_of = part_of or []
+ normalized_part_of = []
+ for part in part_of:
+ if isinstance(part, ResourceSpec):
+ normalized_part_of.append(part)
+ else:
+ normalized_part_of.append(
+ ResourceSpec(self.library, part))
+ self.part_of = normalized_part_of
+
self.modes = kw
def ext(self):
@@ -56,23 +67,8 @@
except KeyError:
# fall back on the default mode if mode not found
return self
-
- def consolidated(self):
- """Get the resource spec in consolidated form.
-
- A resource can be part of a larger resource. Returns the
- resource spec of the consolidated resource, or None if no
- such larger resource exists.
- """
- if self.part_of is None:
- return None
- if isinstance(self.part_of, ResourceSpec):
- return self.part_of
- return ResourceSpec(self.library, self.part_of)
def key(self):
- # XXX has to be a tuple right now as consolidation code depends
- # on this
return self.library.name, self.relpath
def __repr__(self):
@@ -164,29 +160,32 @@
return result
def consolidate(resources):
- consolidated = {}
- processed = []
+ # first map rollup -> list of resources that are in this rollup
+ rollup_to_resources = {}
for resource in resources:
- c = resource.consolidated()
- if c is None:
- processed.append(resource)
- else:
- processed.append(c.key())
- r = consolidated.setdefault(c.key(), [])
- r.append(resource)
+ for rollup in resource.part_of:
+ rollup_to_resources.setdefault(rollup.key(), []).append(resource)
+
+ # now replace resource with rollup consolidated biggest amount of
+ # resources, or keep resource if no such rollup exists
result = []
- for resource in processed:
- if type(resource) is TupleType:
- key = resource
- r = consolidated[key]
- if len(r) == 1:
- result.append(r[0])
- else:
- result.append(r[0].consolidated())
+ for resource in resources:
+ potential_rollups = []
+ for rollup in resource.part_of:
+ potential_rollups.append((len(rollup_to_resources[rollup.key()]),
+ rollup))
+ if not potential_rollups:
+ # no rollups at all
+ result.append(resource)
+ continue
+ sorted_rollups = sorted(potential_rollups)
+ amount, rollup = sorted_rollups[-1]
+ if amount > 1:
+ result.append(rollup)
else:
result.append(resource)
return result
-
+
def render_css(url):
return ('<link rel="stylesheet" type="text/css" href="%s" />' %
url)
Modified: hurry.resource/trunk/src/hurry/resource/interfaces.py
===================================================================
--- hurry.resource/trunk/src/hurry/resource/interfaces.py 2008-09-22 19:41:09 UTC (rev 91366)
+++ hurry.resource/trunk/src/hurry/resource/interfaces.py 2008-09-22 19:49:02 UTC (rev 91367)
@@ -17,6 +17,8 @@
library = Attribute("The resource library this resource is in")
relpath = Attribute("The relative path of the resource "
"within the resource library")
+ part_of = Attribute("A list of potential rollups that this "
+ "resource is part of")
def ext():
"""Get the filesystem extension of this resource.
@@ -37,18 +39,6 @@
If we cannot find a particular mode for a resource, the
resource spec is also used.
"""
-
- def consolidated():
- """Get the resource spec in consolidated form.
-
- A resource can be part of a larger resource, for instance
- multiple CSS files or .js files concatenated to each
- other. This is done for performance reasons to cut down on the
- amount of requests.
-
- Returns the resource spec of the consolidated resource, or
- None if no such larger resource is known.
- """
def key():
"""Returns a unique, hashable identifier for the resource.
More information about the Checkins
mailing list