[Checkins] SVN: zc.buildout/branches/tlotze-download-api/src/zc/buildout/download. download to a temporary location first in order to keep downloads which were aborted in the middle or have checksum mismatches out of the cache

Thomas Lotze tl at gocept.com
Mon Jun 8 13:39:54 EDT 2009


Log message for revision 100750:
  download to a temporary location first in order to keep downloads which were aborted in the middle or have checksum mismatches out of the cache

Changed:
  U   zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.py
  U   zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.txt

-=-
Modified: zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.py
===================================================================
--- zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.py	2009-06-08 12:47:43 UTC (rev 100749)
+++ zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.py	2009-06-08 17:39:54 UTC (rev 100750)
@@ -129,23 +129,33 @@
     def download(self, url, md5sum=None, path=None):
         """Download a file to a given path.
 
-        If path is None, download to a temporary file.
+        The resource is always downloaded to a temporary file and moved to the
+        specified path only after the download is complete and the checksum
+        (if given) matches. If path is None, the temporary file is returned.
 
         See __call__.
 
         """
-        if (self.buildout.get('offline')
-            and urlparse.urlparse(url, 'file').scheme != 'file'):
+        local = urlparse.urlparse(url, 'file').scheme == 'file'
+
+        if self.buildout.get('offline') and not local:
             raise zc.buildout.UserError(
                 "Couldn't download %r in offline mode." % url)
 
         urllib._urlopener = url_opener
-        path, headers = urllib.urlretrieve(url, path)
-        if not check_md5sum(path, md5sum):
+        tmp_path, headers = urllib.urlretrieve(url)
+        if not check_md5sum(tmp_path, md5sum):
+            if not local:
+                os.remove(tmp_path)
             raise ChecksumError(
                 'MD5 checksum mismatch downloading %r' % url)
-        return path
 
+        if path and not local:
+            shutil.move(tmp_path, path)
+            return path
+        else:
+            return tmp_path
+
     def filename(self, url):
         """Determine a file name from a URL according to the configuration.
 

Modified: zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.txt
===================================================================
--- zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.txt	2009-06-08 12:47:43 UTC (rev 100749)
+++ zc.buildout/branches/tlotze-download-api/src/zc/buildout/download.txt	2009-06-08 17:39:54 UTC (rev 100750)
@@ -216,6 +216,13 @@
 This is a foo text.
 >>> ls(cache)
 
+Neither will a cached copy of a resource with a checksum mismatch be created:
+
+>>> download(server_url+'foo.txt', md5('The wrong text.').hexdigest())
+Traceback (most recent call last):
+ChecksumError: MD5 checksum mismatch downloading 'http://localhost/foo.txt'
+>>> ls(cache)
+
 >>> remove(path)
 
 Using namespace sub-directories of the download cache
@@ -394,3 +401,13 @@
 The wrong text.
 >>> cat(cache, 'foo.txt')
 The wrong text.
+
+When trying to download a resource whose checksum does not match, the cached
+copy will neither be used nor overwritten:
+
+>>> write(server_data, 'foo.txt', 'This is a foo text.')
+>>> download(server_url+'foo.txt', md5('The wrong text.').hexdigest())
+Traceback (most recent call last):
+ChecksumError: MD5 checksum mismatch downloading 'http://localhost/foo.txt'
+>>> cat(cache, 'foo.txt')
+The wrong text.



More information about the Checkins mailing list