[ZODB-Dev] Daemon manager design issues

[Toby Dickenson]

One big reply to several posts. Appologies that the attributions are mixe=
d up.

> - There needs to be a way to stop the daemon manager (and the daemon
>   application).  Shall I do this with a signal

The daemon manager needs to handle SIGINT and SIGTERM to shut itself down=
=20
cleanly, because these signals are sent by things outside our control.=20

>  or with a separate
>   utility that talkes to the daemon manager, perhaps through a
>   Unix-domain socket?

This is what daemontools does, and I find that works well.

> - Ditto for restarting the daemon application.  I guess this has to
>   kill the application with a signal

yes, and yes

> unless we want to get fancy and
>   decide on a separate parent-child protocol (which I think would be
>   overkill).  We could do this in three ways:
>
>   - Use a separate utility as described above.

Do you mean that the "daemon manager control" utility communicates with t=
he=20
daemon manager process using a socket (or whatever), then the daemon mana=
ger=20
sends a SIGTERM direct to the application. If so, +1

>   - Send the signal directly to the application (this means there has
>     to be a file with the application's pid; the daemon can write
>     this);

If you mean that the user should signal the managed application directly,=
 then=20
-1. This opens up the same race conditions that Zope/ZEO currently suffer=
s=20
from, because there is a small time window where the pid file on disk is=20
stale during a restart.


> - Logging is configured through environment variables, which are
>   passed on to the application.  Is there a need to be able to
>   configure the manager's logging separately from the application, or
>   is it okay that the manager always logs to the same file as the
>   application?

I suspect this would be useful, but I would be happy provided I can get t=
hem=20
both to target syslog.

> Important: if at any point the application exits with exit status 2,
> it is not restarted.  Any other form of termination (either being
> killed by a signal or exiting with an exit status other than 2) causes
> it to be restarted.

What is the use case for this?=20
(if we need it, then can we make this behaviour optional please?)


> > I'd like to see an approach based on /sbin/init, where
> > something like /etc/inittab lists instances and I can
> > switch Zope 'runlevels' to bring up/down groups of
> > instances.

I think it would make sense to have this implemented as a layer on top of=
=20
Guido's "daemon manager". We could probably even do this as shell scripts=
=2E

> > And the separate utility might have status reporting
> > features, giving a snapshot of servers up/down like
> > ps or top.
>
> Yes, but it won't be able to tell you much more than whether the
> process exists or not=20

I think this should be tristate:
1. Manager process down
2. Manager process up, application down (see below if you dont
   think this state matters)
3. Manager process up, application up


> > [1] basic protection against starting the same instance twice;
> >     the current zdaemon doesn't stop this
>
> Question, how do you know that the same "instance" is already up,
> without building in a lot of knowledge about the application?

Using the status reporting tool. We protect against two copies of the man=
ager=20
process, *not* two copies of the application.

> > [2] You may already do this (I haven't checked your source) but
> >     zdaemon ought to 'cd' into the Zope/var directory to avoid
> >     unnecessarily holding onto directories.
>
> Good point.  The question is, how to decide which directory to chdir
> into without building in too much knowledge about the application.  I
> want this to be fully general.

daemonutils has a working directory for the manager process, which contai=
ns=20
the application start script, its equivalent of our default runlevels, an=
d=20
the manager control socket. I find this approach works well. I think it w=
ould=20
be fair for the manager process to chdir into there, and leave Zope to ch=
dir=20
into Zope/var.


Another nice feature from daemonutils if for the daemon manager process t=
o be=20
able to bring the application down without restarting it, but the manager=
=20
remains running. This can be used to implement the runlevel changing tool=
,=20
provided it is ok to leave the manager processes running in the 'wrong'=20
runlevel.


> > BTW, you may want to look at Dan Bernstein's daemontools for ideas;
> > they provide a framework for starting and stopping daemon processes.
> > (You might even consider using daemontools, but like Bernstein's othe=
r
> > tools the directory organization is a bit eccentric
> Someone posted here earlier saying they were lacking something; I
> forget what.

That was me:
http://lists.zope.org/pipermail/zope-coders/2002-November/002634.html
I can recommend looking at this package for *ideas*, even if you dont end=
 up=20
using the tools.

> > and the license may be a problem.

public domain?