[ZODB-Dev] ZEO client leaking memory?

[Jeremy Hylton]

>>>>> "CW" == Chris Withers <chrisw@nipltd.com> writes:

  CW> Jeremy Hylton wrote:
  >>
  >> We're seeing a very similar problem with an internal project that
  >> uses ZEO.  The client-side consumes so much memory that the
  >> server side fails with a MemoryError.  We haven't made much
  >> progress debugging,

  CW> Is this still the case?

Yes.  The job gets run infrequently and consumes lots of resources, so
I haven't been in a position to do much testing of it.  Once Barry
gets down with Berkeley storage testing, I may have some (machine)
resources available.

  >> It's hard to tell if this is a problem with ZEO or not.  If the
  >> client is leaking memory, the leak could be in any of several
  >> places: in the ClientStorage, in any of the Zope code used by the
  >> client, or in the applications themselves.

  CW> Yup... how do I go about measuring/narrowing it down?

We may be able to isolate ClientStorage vs. application by performing
a storage copy.  Take the FileStorage that contains the unpacked
transaction data generated by the leaky application.  Use the iterator
protocol to copy from the FileStorage to a new storage using ZEO.  If
we see a memory leak, we can blame ZEO because the application isn't
involved.  If we don't see a leak, it's more likely that the
application is leaking.

Jeremy