[Zope] My Zope is leaking memory ... Still no luck. (PythonSc ript bug)

Jean-Francois.Doyon@CCRS.NRCan.gc.ca Jean-Francois.Doyon@CCRS.NRCan.gc.ca
Wed, 18 Dec 2002 12:10:36 -0500


Hello,

Well, I tried to look at the PythonScript issue, and it looks like there's
definitely something "odd" going on here ...

As Jo suggested, I tried "cleaning out" my variables at the end of my Python
Scripts ... I didn't do this for ALL of them though, just the most used
ones, which actually aren't CMF related (Search Engine is one) ...

And now what happens is the memory grows, BUT, at some point, the Zope
process DIES with a SIG11 (Aieeee!), restarts, and voila, of course, the
memory is clean.

This I suppose is a blessing is disguise, as I don't have to keep an eye on
it anymore, I can actually rely on a SIG11 occuring and restarting the
leaking process for me !!!

But this to me indicates some worrying problems with the PythonScript
implementation. Although I have looked everywhere else for the source of my
memory leak, this is the best candidate yet. I even looked at this document:
http://www.zope.org/Members/mcdonc/HowTos/PreventAndDebugMemLeaks and looked
through my scripts for global variables that don't get reset, and circular
references, but not luck ...

I am confused about the following statements though:

"DO use REQUEST.set and REQUEST.__setitem__ sparingly."
"DONT stuff things that are acquisition-wrapped into REQUEST or into a
session, or into any other object that is transient."

What does that mean exactly? :) I don't use SESSION's , but I like to stuff
things into my REQUEST object indeed quite a bit. Could this be it? And if
so ... out of pure curiosity, why is this bad? I have found using dtml-let
to be awkward to use at times ...

and

"DO use global variables sparingly."
"DONT stuff things into a global variable without later clearing or deleting
the variable."

Global in reference to what ? :) In my External Methods, I have no global
variables, that is no variables that don't belong inside a def'ed function
(and allways "reset" before being used) ...
In PythonScripts themselves, I think I NEVER use functions, just straight
code, so by definition all variables ARE global to that script ... Is this
bad?

Thanks again to  everyone for the help and comments!

J.F.

-----Original Message-----
From: Jo Meder [mailto:jo@meder.de]
Sent: Friday, November 29, 2002 5:29 PM
To: Jean-Francois.Doyon@ccrs.nrcan.gc.ca
Cc: zope@zope.org
Subject: Re: [Zope] My Zope is leaking memory ... Still no luck.


Am 29.11.2002, 21:06 Uhr
	schrob Jean-Francois.Doyon@CCRS.NRCan.gc.ca
<Jean-Francois.Doyon@CCRS.NRCan.gc.ca>:

[Forcing transient objects to be deleted at end of PythonScripts to
 avoid memory leaks]

> Really? Wow, you'd think such a required behavior would be documented ...
> Unless I missed it somewhere.

AFAIK it isn't documented and IMHO it's a bug anyway, so I suppose it
should not be documented but solved. Sigh. So much to do and so little
time.

> I do indeed use quite a bit of Python Scripts ... Problem is many of them
> are part of the CMF ... 

Perhaps we have a pattern here. Just like you we use CMF, but we started
with an older version (that was mostly DTML and only some PythonScripts)
and have since then rewritten or at least changed almost everything in
the skins folders.

One problem wrt to the memory leaks we have not solved yet: When we
refresh one of our products that is part of our CMS Zope starts to leak
memory like there's no tomorrow. Restarting Zope immediately stops this.

> I really don't want to have to go through every bit of python script I
> have to do this, and would much prefer finding the root cause!

If you have any success, even clues or insights: I'd very much
appreciate it if you kept me (or the list, of course) posted.


	Jo.


-- 
Internetmanufaktur Jo Meder ---------------------- Berlin, Germany
http://www.meder.de/ ------------------- fon: ++49-30-417 17 63 33
Kollwitzstr. 75 ------------------------ fax: ++49-30-417 17 63 45
10435 Berlin --------------------------- mob: ++49-170- 2 98 89 97
Public GnuPG-Key ---------- http://www.meder.de/keys/jo-pubkey.txt