[ZODB-Dev] Errors running a Stress Test

[Brandon Ehle]

Tim Peters wrote:
> [Brandon Ehle]
> 
>>I've used ZODB w/ ZEO in the past, and I wrote a quick stress test to
>>try and test for bugs that I've encountered in the past.  I'm
>>currently running into lots of bugs that I haven't seen before,
> 
> 
> You showed exactly two tracebacks later, one on the server and one on the
> client.  How does that relate to "lots"?  For example, you have many
> instances of these same two tracebacks?  You have many distinct tracebacks,
> but tired of collecting them?  Something else?
> 

When exceptions start happening for me, they repeat ALOT.  There were 
some other exceptions, but by the time I could grab them, they scrolled 
right off my 10MB limit.

> 
>>so perhaps I'm doing something wrong that I didn't notice, but I was
>>curious if anyone else was seeing these problems.
> 
> 
> We've gotten little feedback on 3.3a2 so far, and (of course) it is an alpha
> release.  Thanks for testing it!
> 
> 
>>After running for a couple hours I got all sorts of errors.  Am I
>>doing something obviously wrong, or am I just tickling ZODB in the
>>wrong way?
> 
> 
> They're probably bugs.  *Starting* the database isn't robust:
> 
> 
>>    conn = db.open()
>>    if 'test' not in conn.root():
>>        conn.root()['test'] = OOBTree()
>>        get_transaction().commit()
> 

Originally I was only running 1 thread, which is probably why I never 
noticed.

> 
> because when you first run it, all the threads may try to create the root's
> 'test' simultaneously.  Indeed, that's what happened when I first tried it,
> leaving only one thread surviving to continue beyond that point.  The rest
> looked fine to me, though.
> 
> 
>>***** Client side errors
>>...
>>  File "C:\Python23\Lib\site-packages\ZEO\cache.py", line 725, in
>>    _makeroom size, e = self.filemap.pop(ofs)
>>KeyError: 301258
> 
> 
> Probably a bug in the new ZEO client cache.
> 
> 
> 
>>***** Server side errors
>>
>>2004-03-15T11:46:52 ERROR(200) Conflict Resolution Unexpected error
>>Traceback (most recent call last):
>>  File "C:\Python23\Lib\site-packages\ZODB\ConflictResolution.py",
>>line 108, in tryToResolveConflict
>>    old = state(self, oid, oldSerial, prfactory)
>>  File "C:\Python23\Lib\site-packages\ZODB\ConflictResolution.py",
>>line 38, in state
>>    p = p or self.loadSerial(oid, serial)
>>  File
>>"C:\Python23\Lib\site-packages\ZODB\FileStorage\FileStorage.py", line
>>    581, in loadSerial raise POSKeyError(oid)
>>POSKeyError: 0000000000000001
> 
> 
> Probably some other bug <wink>.
> 
> 
>>Win2K SP4
>>Python-2.3.3
>>ZODB-3.3a2
> 
> 
> If you can, try running from current CVS HEAD of the ZODB module.  Bugs have
> certainly been fixed since 3.3a2 was released, although it would take more
> time than I've got to guess whether they account for your symptoms.
> 

I'll give that a whirl today.

> I've been running your test for about half an hour now on ZODB HEAD without
> incident, but with server and clients all on a single machine (that's all I
> can get at now), also on Windows.
> 
> Oops!  I saw one akin to your server-side traceback whiz by while typing
> that.  Nothing odd on the clients yet.
> 

I left this running over the weekend and when I got in Monday is when I 
noticed all the exceptions whizzing by.  It was fine when I left on 
Friday (it had been running for 2 hours maybe).