[Checkins] SVN: zc.resumelb/branches/jinty-fix-worker-shutdown-race/src/zc/resumelb/ Fixed: A race condition when multiple workers are quickly shutdown

[Brian Sutherland]

Log message for revision 128074:
  Fixed: A race condition when multiple workers are quickly shutdown
  when used with ZooKeeper and single-version mode. An uncaught
  NoNodeException was raised when the loadbalancer tried to get the
  properties of a node already shut down.
  

Changed:
  U   zc.resumelb/branches/jinty-fix-worker-shutdown-race/src/zc/resumelb/README.txt
  U   zc.resumelb/branches/jinty-fix-worker-shutdown-race/src/zc/resumelb/zk.py

-=-
Modified: zc.resumelb/branches/jinty-fix-worker-shutdown-race/src/zc/resumelb/README.txt
===================================================================
--- zc.resumelb/branches/jinty-fix-worker-shutdown-race/src/zc/resumelb/README.txt	2012-10-19 15:21:57 UTC (rev 128073)
+++ zc.resumelb/branches/jinty-fix-worker-shutdown-race/src/zc/resumelb/README.txt	2012-10-19 15:35:17 UTC (rev 128074)
@@ -242,6 +242,14 @@
 Change History
 ==============
 
+0.x.x (UNRELEASED)
+------------------
+
+- Fixed: A race condition when multiple workers are quickly shutdown
+  when used with ZooKeeper and single-version mode. An uncaught
+  NoNodeException was raised when the loadbalancer tried to get the
+  properties of a node already shut down.
+
 0.7.1 (2012-10-17)
 ------------------
 

Modified: zc.resumelb/branches/jinty-fix-worker-shutdown-race/src/zc/resumelb/zk.py
===================================================================
--- zc.resumelb/branches/jinty-fix-worker-shutdown-race/src/zc/resumelb/zk.py	2012-10-19 15:21:57 UTC (rev 128073)
+++ zc.resumelb/branches/jinty-fix-worker-shutdown-race/src/zc/resumelb/zk.py	2012-10-19 15:35:17 UTC (rev 128074)
@@ -208,12 +208,16 @@
     if options.single_version:
         @addrs
         def get_addrs(a):
-            to_send[0] = dict(
-                (zc.parse_addr.parse_addr(addr),
-                 zk.get_properties(
-                     path+'/workers/providers/'+addr).get('version')
-                 )
-                for addr in addrs)
+            r = {}
+            for addr in addrs:
+                try:
+                    version = zk.get_properties(
+                            path+'/workers/providers/'+addr).get('version')
+                except zookeeper.NoNodeException:
+                    # If nodes are shut down in quick succession we can get here
+                    continue
+                r[zc.parse_addr.parse_addr(addr)] = version
+            to_send[0] = r
             awatcher.send()
     else:
         @addrs