[Zope-dev] ERROR(200) ZODB Couldn't load state for... Memory problem?

Russ Ferriday russf at topia.com
Sun Jul 25 12:43:19 EDT 2004


Well, I can understand Tim's comment about droppings. This 
configuration has made a mockery of my weekend. >:-(
Looking on the bright side - it'll soon be Monday ;)

First an interesting data point.

Assume for now that I'm not naming my zeo-clients - this would be 
begging for disaster - but all is not lost. For some arcane reason I 
have not yet tracked down, after running my 2 clients under load, I get 
these...
-rw-r--r--  1 plone  staff  10006534 25 Jul 05:30 
/usr/local/zope/instances/prodZope/etc/1-None-0
-rw-r--r--  1 plone  staff   1404198 25 Jul 05:37 
/usr/local/zope/instances/prodZope/etc/1-None-1
-rw-r--r--  1 plone  staff   5723516 25 Jul 05:02 
/usr/local/zope/instances/prodZope/log/1-None-0
-rw-r--r--  1 plone  staff  10043763 25 Jul 04:20 
/usr/local/zope/instances/prodZope/log/1-None-1

Nowhere have I asked it for cache files in either etc or log, but it 
sure makes sense that zope conspires to spread  them around like this.

Next for a little scat perusal of my own. We're talking 2.7.2 now.

Here's the mystery.

I have never seen a cache file, in any of the recent 3 Zope versions 
I've just tested, with a name other than *-None-*. Has anyone else?

Note as well for anyone following on the Multi-Processor path, that 
Tres's tip about clienthome was slightly off, and yes, Paul Winkler was 
nearer the mark.

Tres wrote..
> # zope.conf.appserver1
> %include zope.conf.common
> clienthome /path/to/zope/var/appserver1

Clienthome has no bearing on the cache file location. (AFAICT)

On 23 Jul 2004, at 18:38, Paul Winkler wrote:

> You mean this, from zope.conf.in?
>
> # Directive: zeo-client-name

Yes, Paul. That's the way to go. And it should be pretty easy. Add a 
single line to each conf file and restart.
Not so. I never saw the names of my cache files change. And they stayed 
in etc, and log sub dirs.

I walk through it here...

Starting in /ClientCache.py", line 142, at my pdb_trace() I believe 
that the client param should be the value of zeo-client-name from my 
.conf file.  But I'm seeing 'None'. So we never even try to make 
anything by temporary caches.

Here's my stack:
   File "/usr/local/zope/zope/lib/python/Zope/Startup/run.py", line 50, 
in ?
     run()
   File "/usr/local/zope/zope/lib/python/Zope/Startup/run.py", line 19, 
in run
     start_zope(opts.configroot)
   File "/usr/local/zope/zope272/lib/python/Zope/Startup/__init__.py", 
line 51, in start_zope
     starter.startZope()
   File "/usr/local/zope/zope272/lib/python/Zope/Startup/__init__.py", 
line 230, in startZope
     Zope.startup()
   File "/usr/local/zope/zope272/lib/python/Zope/__init__.py", line 47, 
in startup
     _startup()
   File "/usr/local/zope/zope272/lib/python/Zope/App/startup.py", line 
57, in startup
     DB = configuration.dbtab.getDatabase('/', is_root=1)
   File "/usr/local/zope/zope272/lib/python/DBTab/DBTab.py", line 96, in 
getDatabase
     db = self._createDatabase(name, is_root)
   File "/usr/local/zope/zope272/lib/python/DBTab/DBTab.py", line 113, 
in _createDatabase
     db = factory.open()
   File "/usr/local/zope/zope272/lib/python/Zope/Startup/datatypes.py", 
line 172, in open
     DB = self.createDB()
   File "/usr/local/zope/zope272/lib/python/Zope/Startup/datatypes.py", 
line 169, in createDB
     return ZODBDatabase.open(self)
   File "/usr/local/zope/zope272/lib/python/ZODB/config.py", line 97, in 
open
     return ZODB.DB(section.storage.open(),
   File "/usr/local/zope/zope272/lib/python/ZODB/config.py", line 148, 
in open
     read_only_fallback=self.config.read_only_fallback)
   File "/usr/local/zope/zope272/lib/python/ZEO/ClientStorage.py", line 
302, in __init__
     client=client, var=var)
   File 
"/usr/local/zope/zope/lib/python/ZEO/ClientCache.py"(142)__init__()
-> import pdb; pdb.set_trace();

up to 
/usr/local/zope/zope272/lib/python/ZEO/ClientStorage.py(302)__init__()
where I see formal param of client=None, and useful confirmation that 
client is "A name used to construct persistent cache filenames.         
    Defaults to None"
There's no assignment to client in __init__(). So why is a client name 
not being passed in?

up to /usr/local/zope/zope272/lib/python/ZODB/config.py(148)open()
where I see client=self.config.client being passed to the constructor 
of class ZEOClient(BaseConfig)
and do :
self.config.__dict__
{'read_only': False, 'realm': None, 'name': 'zeostorage', 'storage': 
'1', '_matcher': <SectionMatcher for type 'zeoclient'>, 
'min_disconnect_poll': 5, 'server': [<ZConfig.datatypes.SocketAddress 
instance at 0x20bdbc0>], 'cache_size': 20000000, 'client': None, 'var': 
'/usr/local/zope/instances/zeo/var', 'max_disconnect_poll': 300, 
'_name': None, 'read_only_fallback': False, 'wait': True}
All fine, except 'client': None.

Interpretation gets a bit messy from there, so I switch to a top-down 
approach:
Starting from my config file:
grep zeo-client-name zope_1.conf
# Directive: zeo-client-name
#     zeo-client-name.  If you use ZEO and you don't set a
#     zeo-client-name, the client cache is stored in temporary files
#     of zeo-client-name is used to uniquely identify the local cache
#    zeo-client-name zeo1
zeo-client-name zeo1

Fair enough. I'm setting a name. Here are my other critical items:
zodb_db temporary>
     <temporarystorage>
       name sessions
     </temporarystorage>
     mount-point /temp_folder
     container-class Products.TemporaryFolder.TemporaryContainer
  </zodb_db>

<zodb_db main>
    mount-point /
    <zeoclient>
      server localhost:8800
      storage 1
      name zeostorage
      var $ZEO_INSTANCE/var
    </zeoclient>
  </zodb_db>

So I cd to the zope root and..
grep -r zeo-client-name *
lib/python/Zope/Startup/zopeschema.xml:  <key name="zeo-client-name" 
handler="zeo_client_name">

So zeo-client-name is handled by zeo_client_name. Great! I shoved a 
pdb_trace there. n and l  give me
76     def zeo_client_name(value):
  77         value and _setenv('ZEO_CLIENT', value)
  78         import pdb; pdb.set_trace()
  79  ->     return value
(Pdb) value
'zeo1'

Cool! So far so good.

But this brings me to the gap..  How is ZEO_CLIENT in the environment 
going to get into self.config.client in the ZEOClient? Bearing in mind 
grep -r ZEO_CLIENT * shows me only comments, apart from the handler, 
where it is set.

Any ideas? Anyone seen anything but *-None-* for cache files?

Meanwhile, I'm going to see why zope gets creative about spreading the 
cache files around.

--r.

On 23 Jul 2004, at 21:37, Chris McDonough wrote:

> On Fri, 2004-07-23 at 16:21, Tim Peters wrote:
>> [Chris McDonough]
>>> ...
>>>          self._f[current] = open(self._p[current],'w+b')
>>>
>>> .... will be likely to fail at the last line if you're using
>>> nonpersistent cache files, because self._p[current] is (bogus)
>>> '1-None-0' (relative bogus filename).
>>
>> Is it really *likely* to fail?
>
> I suppose it depends on the working directory of the shell/process used
> to start Zope.  Zope doesn't mess with the working directory on its 
> own,
> AFAIK.
>
> If you follow Richard Stevens' ("UNIX Network Programming" guy,
> apparently now dead) advice, he says that "well-behaved" daemon
> processes should change their working directory to "/".  So I suspect
> there are daemonizers that do this.
>
> Guido's zdrun daemon (which "zopectl" uses) gives you an option to set
> the working directory of the daemonized process, but I don't use it
> (neither zdrun nor the option, that is).  It does nothing to the 
> working
> directory by default.
>
> But I think the common case is that the program is run out of an
> /etc/init.d "rc" script, and I suspect the working directory is "/" 
> when
> Zope gets started in that circumstance.  Which I guess makes the error
> understandable.
>
>> It's just a name, and it's opened in
>> 'w' ('+b') mode, not 'r' mode.  That is, it creates the file -- no
>> file of that name need already exist (and if one does, it tries to
>> overrwrite it).  Running on Windows most days, I'm not usually aware
>> of all the permission bugs Linuxheads delight in torturing themselves
>> with <wink>.
>
> Yes.  Gotta agree with you there.  I don't think a day passes where I
> don't want to rip the face off the guy who proclaimed that TCP ports
> below 1024 couldn't be bound to by a user other than root.  What a
> disaster.
>
>>> There should probably be a _using_persistent_cache flag attr rather 
>>> than
>>> trying to inspect self._p to find out if we're using persistent 
>>> caches.
>>
>> +1.  As you later discovered, this "hmm, let's try to guess what we're
>> doing based on obscure droppings" business is a continuing bug
>> factory.
>
> Thankfully, Dieter fixed it so it doesn't (at least in this one case).
>
>>
>>> I may try to work up a patch + test for this later.
>>
>> I'm neutral on whether you try, but +1 on you actually doing it 
>> <wink>.
>
> Too late!  It's already fixed.  I didn't know either. ;-)  This thread
> was full of sound and fury....
>
> - C
>
>
> _______________________________________________
> Zope-Dev maillist  -  Zope-Dev at zope.org
> http://mail.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope )
>
—————————————————————
Russ Ferriday
Solution Workshops for Plone
(+44) (0) 7789 338868
http://www.solutionworkshops.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 9748 bytes
Desc: not available
Url : http://mail.zope.org/pipermail/zope-dev/attachments/20040725/d3b6ea40/attachment.bin


More information about the Zope-Dev mailing list