[ZODB-Dev] Webkit Threading and ZODB 3.3a2: problems on Windows

[Tim Peters]

[Matt Feifarek, w/ some ugly thread symptoms seen only on Windows]

Matt, what does this print for you if you run it on your Linux box?

"""
import thread, time

def worker():
    print thread.get_ident()

for i in range(10):
    thread.start_new_thread(worker, ())
    time.sleep(1)
"""

A typical run on Windows prints the same number 10 times.  If it typically prints 10 distinct numbers for you, I'm betting that's a clue.

There are no deliberate ways in which Python thread semantics differ across Linux and Windows.  There are huge pragmatic differences, though, due to the way the OSes implement and schedule their native threads.  The one most often "to blame" when platform-specific behavior is seen is that Windows is typically much more willing to switch threads frequently than Linux.  But Windows is also eager to reuse internal handles ASAP (thread.get_ident() is the return value from the Win32 API GetCurrentThreadId() call on Windows, and is used by ZODB as a key in a dict to map the current thread to its current transaction), while I *expect* Linux is not (thread.get_ident() is the return value from the pthreads pthread_self() on Linux, and Linux is an oddball among Unixes in confusing thread ids with process ids -- most Unixes don't reuse pids for a loooong time, but Windows reuses tids ASAP).

I don't know that this relates to the problem you're seeing -- just trying to get more evidence now, and have been nervous about the thread-id -> transaction dict since I first saw it.

If you let a thread die with changes in progress (neither commit nor abort on its then-current transaction), it looks all but certain to me that ZODB's tid->transaction dict will get confused by the next thread Windows creates (which is all but certain to have the same tid as the thread that just died).