[Zope] To thread or not to thread ...
Mon, 24 Jun 2002 10:11:47 +0200
I never done this before, but I think you should take care
about transaction commit which is done after each request.
So if you start a new thread, you probably have to get a new
connection to ZODB, do what you need to do, and commit it.
On Sat, Jun 22, 2002 at 06:11:43AM -0700, Terry Hancock wrote:
> Hi all,
> As yet another iteration in a very image intensive product,
> I'm trying to build a tree of images. They are generated
> procedurally, and may differ from installation to installation
> (they don't currently, but they probably will).
> So it's much more convenient to generate them procedurally
> in the manage_add<my-product>() function than to store them
> in, say, a .zexp file. (I also tried making them on-the-
> fly, but that's a whole 'nother story).
> Nevertheless, it takes quite awhile for PIL to chunk
> through all those images (currently 2880).
> If I just use a straight iteration, the browser will
> time-out during the installation process, which is kind
> of ugly, and will no doubt freak out my potential user/
> administrator. Also, it's not absolutely clear to me
> that Zope won't cut me off somehow if my request takes
> too long. (Any reassurances/warnings on this point?).
> Alternatively, I can put components of the installation
> process into separate threads. This nicely drops me
> back into the management interface and runs the object-building
> process in the background. However, I then have to think
> about checking it to make sure it's done (probably I can
> handle this, though I haven't yet -- I just check it
> manually later).
> The big headache though, is that with the threaded version,
> I have folders disappear out of the management interface!
> If I wait long enough, eventually they just start to
> disappear. Same thing if I restart. So the persistence
> machinery must not be working in the threaded case.
> But why not?
> The code is essentially the same, except that in the
> non-threaded version I just call the function, whereas
> in the threaded version, I create a thread to run it
> with threading.Thread(). I do this at the highest level
> in the loop, which means about 4 threads should be started,
> which I wouldn't think would tax my system (an earlier
> version with about 50 threads did cause a massive slowdown).
> Both versions use ordinary 'Folder' objects and create
> them using the Zope API with manage_addFolder() calls
> on the parent folders.
> Should I abandon threading here? (and just document
> this in the INSTALL file). Should I use full-fledged
> processes? Or am I just forgetting to do something?
> (I've done almost no threaded programming before this --
> I did run a threaded test version of this code that
> generated the files on the filesystem outside of
> Zope. All my tests with that version went fine.).
> Thanks in advance for any ideas,
> Terry Hancock
> Anansi Spaceworks
> P.O. Box 60583
> Pasadena, CA 91116-6583
> Zope maillist - Zope@zope.org
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-dev )