[ZODB-Dev] all webserver threads blocking on db.open()

[Claudiu Saftoiu]

Hello all,

I'm using Repoze.BFG, with paster to launch the webserver. This is a
similar issue to the one I emailed about before titled
"server stops handling requests - nowhere near 100% CPU or Memory
used"

The situation is the same. I used z3c.deadlockdebugger , and what
I notice is that, when the server is blocked, there are about 100
threads running (as opposed to 15 or so when the server has
just started), and all their stack traces look like this:

Thread 140269004887808:
  File "/usr/lib/python2.6/threading.py", line 504, in __bootstrap
    self.__bootstrap_inner()
  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.6/threading.py", line 484, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/tsa/env/lib/python2.6/site-packages/paste/httpserver.py",
line 878, in worker_thread_callback
    runnable()
  File "/home/tsa/env/lib/python2.6/site-packages/paste/httpserver.py",
line 1052, in <lambda>
    lambda: self.process_request_in_thread(request, client_address))
  File "/home/tsa/env/lib/python2.6/site-packages/paste/httpserver.py",
line 1068, in process_request_in_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python2.6/SocketServer.py", line 322, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python2.6/SocketServer.py", line 617, in __init__
    self.handle()
  File "/home/tsa/env/lib/python2.6/site-packages/paste/httpserver.py",
line 442, in handle
    BaseHTTPRequestHandler.handle(self)
  File "/usr/lib/python2.6/BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "/home/tsa/env/lib/python2.6/site-packages/paste/httpserver.py",
line 437, in handle_one_request
    self.wsgi_execute()
  File "/home/tsa/env/lib/python2.6/site-packages/paste/httpserver.py",
line 287, in wsgi_execute
    self.wsgi_start_response)
  File
"/home/tsa/env/lib/python2.6/site-packages/repoze/zodbconn/connector.py",
line 18, in __call__
    conn = self.db.open()
  File "/home/tsa/env/lib/python2.6/site-packages/ZODB/DB.py", line 729, in
open
    self._a()
  File "/usr/lib/python2.6/threading.py", line 123, in acquire
    rc = self.__block.acquire(blocking)

The server gets to a blocked state every 24 hours or so. Simply restarting
the webserver works fine; however, i'd like to know what the problem is so
restarting won't be necessary, and to prevent it from getting worse. Any
ideas/
suggestions?

Thanks,
- Claudiu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.zope.org/pipermail/zodb-dev/attachments/20120507/73fd863b/attachment.html>