[Checkins] SVN: Sandbox/J1m/resumelb/src/zc/resumelb/zk. Fixed bug running worker in non-test mode.

[Jim Fulton]

Log message for revision 124267:
  Fixed bug running worker in non-test mode.
  
  Added tests to make sure that worker and lb work in non-test mode.
  
  Removed promise of unimplemented feature to allow no access log.
  
  Added ability to pass an object as the access log (mainly for simulation).
  

Changed:
  U   Sandbox/J1m/resumelb/src/zc/resumelb/zk.py
  U   Sandbox/J1m/resumelb/src/zc/resumelb/zk.test

-=-
Modified: Sandbox/J1m/resumelb/src/zc/resumelb/zk.py
===================================================================
--- Sandbox/J1m/resumelb/src/zc/resumelb/zk.py	2012-01-31 10:37:01 UTC (rev 124266)
+++ Sandbox/J1m/resumelb/src/zc/resumelb/zk.py	2012-01-31 12:50:09 UTC (rev 124267)
@@ -38,7 +38,7 @@
     worker.zk = zk
     if run:
         try:
-            worker.server.run_forever()
+            worker.server.serve_forever()
         finally:
             logging.getLogger(__name__+'.worker').info('exiting')
             zk.close()
@@ -65,7 +65,7 @@
     parser.add_option(
         '-l', '--access-log', default='-',
         help='Access-log path.\n\n'
-        'Use - (default) for standard output and an empty string to suppress.\n'
+        'Use - (default) for standard output.\n'
         )
     parser.add_option(
         '-b', '--backlog', type='int',
@@ -141,8 +141,11 @@
         spawn= gevent.pool.Pool(options.max_connections)
     else:
         spawn = 'default'
-    accesslog = (sys.stdout if options.access_log == '-'
-                 else open(options.access_log, 'a'))
+
+    accesslog = options.access_log
+    if isinstance(accesslog, str):
+        accesslog = sys.stdout if accesslog == '-' else open(accesslog, 'a')
+
     server = gevent.pywsgi.WSGIServer(
         addr, lb.handle_wsgi, backlog = options.backlog,
         spawn = spawn, log = accesslog)

Modified: Sandbox/J1m/resumelb/src/zc/resumelb/zk.test
===================================================================
--- Sandbox/J1m/resumelb/src/zc/resumelb/zk.test	2012-01-31 10:37:01 UTC (rev 124266)
+++ Sandbox/J1m/resumelb/src/zc/resumelb/zk.test	2012-01-31 12:50:09 UTC (rev 124267)
@@ -112,8 +112,7 @@
                             Address to listed on for web requests
       -l ACCESS_LOG, --access-log=ACCESS_LOG
                             Access-log path.  Use - (default) for
-                            standard output and an empty string to
-                            suppress.
+                            standard output.
       -b BACKLOG, --backlog=BACKLOG
                             Server backlog setting.
       -m MAX_CONNECTIONS, --max-connections=MAX_CONNECTIONS
@@ -176,12 +175,15 @@
     127.0.0.1 - - [2012-01-28 15:29:30] "GET /hi.html HTTP/1.0" 200 226 0.002349
     HTTP/1.0 200 OK...
 
-If we create another worker, it will be seen by the load balancer:
+If we create another worker, it will be seen by the load
+balancer. This time, we're not going to run the worker in test mode.
+We want to make sure that non-test mode works:
 
-    >>> worker2 = zc.resumelb.zk.worker(
+    >>> worker2_greenlet = gevent.spawn(zc.resumelb.zk.worker,
     ...     app, None,
     ...     zookeeper='zookeeper.example.com:2181', path='/test/lb/workers',
-    ...     address='127.0.0.1:0', run=False)
+    ...     address='127.0.0.1:0')
+    >>> gevent.sleep(.01)
 
     >>> len(lb.pool.workers)
     2
@@ -240,14 +242,16 @@
     >>> print sock.recv(9999) # doctest: +ELLIPSIS
     HTTP/1.0 200 OK...
 
+    >>> sock.close()
+    >>> sock2.close()
+
 We didn't get an access log entry in the output, because it's in the
 access log.
 
     >>> accesslog.flush()
     >>> with open('access.log') as f:
-    ...     print f.read()
+    ...     print f.read(),
     127.0.0.1 - - [2012-01-29 14:11:37] "GET /hi.html HTTP/1.0" 200 226 0.001074
-    <BLANKLINE>
 
 By looking at the lb's pool's skilled data structure, we can see that
 the test request classifier was used.
@@ -255,13 +259,60 @@
     >>> list(lb.pool.skilled)
     ['a', 'b', "yup, it's a test"]
 
+Let's shut down the workers:
+
+    >>> worker.stop()
+    >>> worker.zk.close()
+    >>> worker2_greenlet.kill()
+    >>> gevent.sleep(.01)
+
+We see that there are fewer workers:
+
+    >>> len(lb.workletts)
+    0
+
+
+Not that if we looked at the number of pools, it would still be
+2. This is an artifact of the way the test is run.  We shutdown/killed
+the servers, but we didn't close the open worker connections.  We
+don't have the plumbing to do that and it's only an issue for the tests.
+
 OK. now let's shut down the server and lb.
 
     >>> server.stop()
     >>> lb.stop()
     >>> lb.zk.close()
 
-And the workers:
+Finally, let's test that:
 
-    >>> worker.stop()
-    >>> worker2.stop()
+- The server runs in non-test mode and that
+
+- We can pass an object as the access log parameter.
+  (Used by simulation script.)
+
+    >>> worker_greenlet = gevent.spawn(zc.resumelb.zk.worker,
+    ...     app, None,
+    ...     zookeeper='zookeeper.example.com:2181', path='/test/lb/workers',
+    ...     address='127.0.0.1:0')
+
+    >>> import StringIO
+    >>> accesslog = StringIO.StringIO()
+
+    >>> lb_greenlet = gevent.spawn(zc.resumelb.zk.lbmain, [
+    ...     '-a127.0.0.1:0',
+    ...     '-l', accesslog, 'zookeeper.example.com:2181', '/test/lb'])
+    >>> gevent.sleep(.1)
+
+    >>> [addr] = map(zc.parse_addr.parse_addr,
+    ...              zk.get_children('/test/lb/providers'))
+    >>> sock = gevent.socket.create_connection(addr)
+    >>> sock.sendall('''GET /hi.html HTTP/1.0\r
+    ... Host: h1.com\r
+    ... Content-Length: 0\r
+    ... \r
+    ... ''')
+    >>> print sock.recv(9999) # doctest: +ELLIPSIS
+    HTTP/1.0 200 OK...
+
+    >>> print accesslog.getvalue(),
+    127.0.0.1 - - [2012-01-31 07:46:31] "GET /hi.html HTTP/1.0" 200 226 0.001165