Linux

Use runit to supervise Linux services

Gerrit Pape's runit is similar to djb's daemontools and allows you to run supervised services, like daemontools, and also allows you to replace SysV init completely if you so desire. Vincent Danen tells you how to get started with runit.

Traditional Linux systems use SysV init for starting the system: running init scripts to start and stop services and also to manage system runlevels and one-time startup scripts. The problem with using a SysV init system is that services are also treated as one-time startup scripts. When init starts the system and calls a service, such as sshd, the service will start; but if sshd dies for some reason, it will remain down until an administrator issues a service sshd start command to restart the service.

This can be mitigated to some degree by starting services like sshd directly from /etc/inittab, similar to getties for login. All the console logins are separate getties, and init restarts them when users log out. Running all services out of /etc/inittab has its own shortcomings, however, as they cannot be easily stopped without rebooting.

Gerrit Pape's runit is similar to djb's daemontools and allows you to run supervised services, like daemontools, and also allows you to replace SysV init completely if you so desire.

Some distributions provide runit, some do not. If your chosen distribution does not, runit is easy to download and compile.

runit can still be used without replacing init and it works just fine, with minimal effort. To begin, create a script called /sbin/runsvdir-start with the following:

#!/bin/sh

PATH=/command:/usr/local/bin:/usr/local/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin

exec env - PATH=$PATH
runsvdir -P /service 'log: ...........................................................................................................................................................................................................................................................................................................................................................................................................'

This is the script that init will start. What it does is call the runsvdir program on the /service directory; this is the directory where all supervised services will be linked in order to tell runsvdir what to manage.

Next, edit /etc/inittab and add:

SV:123456:respawn:/sbin/runsvdir-start

to the bottom of the file. This will start (and re-start, if it ever exits) the /sbin/runsvdir-start program.

Next, you need to create run scripts for supervised services. This is quite easy; a number of run scripts can be found on the runit Web site. For instance, to create a supervised sshd, create the directory /var/service/sshd and inside it create a file called ./run; this is the run script. It should have the following contents:

#!/bin/sh
exec /usr/sbin/sshd -D >/dev/null 2>&1

Make sure the run script is mode 0750 so that it is executable. Next, create a symbolic link of /var/service/sshd into the /service directory:

# cd /service
# ln -s /var/service/sshd .

Within a few seconds, runsvdir will notice the new symbolic link, create any control files that are required, and will execute the ./run script. If the script terminates, runsvdir will re-execute the ./run script until told to stop. The sv program can control services; using sv start /service/sshd will bring up the service (if down), and sv stop /service/sshd will stop the service. Otherwise, runsvdir will always ensure the service is up.

Obviously there is a lot more to using supervised services; you can create logging services that will take the output of the service and write it to log files. In the above example we simply redirected the output to /dev/null, letting sshd log to syslog instead. In that case, a log service is not necessary, but on any service that has output, be it to stderr or stdout, a logging service can be used to capture and log it, regardless of whether it logs to syslog as well. We'll take a look at logging services next week.

Download tip as a PDF.

About

Vincent Danen works on the Red Hat Security Response Team and lives in Canada. He has been writing about and developing on Linux for over 10 years and is a veteran Mac user.

9 comments
GimbalOx
GimbalOx

I like the thought of using DJ Bernstein's supervise (or something similar) to control execution of system-critical services (such as mail, DNS, and also HTTP proxy i.e. squid, as well as the jackd audio system server, on my home network gateway). However, I'm concerned about the prospect of using runit, given that I'm not sure how it would be used in a situation like the following: Process A is started during init, and must be monitored by the 'supervise' process Process B is started during init, and must also be monitored by the 'supervise' process Process B requires that process A will have been started, before process B can be started successfully It could happen, as in the case: - Process A is a PostgreSQL server - Process B is an EmForge instance ( http://emforge.org ) or a JamWiki instance ( http://jamwiki.org/ ) that would have been configured to use PostgreSQL on localhost I guess that something could be kludged in B's "run" script, to make the script sleep until (1) something like DJB's 'svstat', or some other service-availability-checking tool, would indicate that process A is up and running successfully, then start B, or else (2) abort, after some predtermined number of seconds will have expired. In the case of SysV init, you could do this by giving the init script for process A a higher effective init "priority" than that for process B -- e.g name the links to the respective init scripts like so, /etc/rc2.d/S10processA and /etc/rc2.d/S20processB -- but, of course, even then, SysV init might try to start process B without process A having been started successfully. It only matters that the previous init script A will have *exited*, before the init script B will be executed -- whether or not script A will have exited with a code indicating "success". Considering that (after having thought this out, in writing, after all) I guess that runit could appear to allow the administrator to make an even more fine-grained control over the the init cycle. Secondly, my concern is in that a number of SysV init scripts for certain packages (as distributed from Debian, for example) might be found to contain some "special code" - e.g. there's a little of some instance-specific stuff in /etc/init.d/apache2 - such that could ideally be ported for use in the service's runit script -- but what about when the upstream package maintainer changes the contents of the original init script? What would keep one's own runit script synchrnized with the upstream init script? Nothing but oneself, indeed... Considering how extensively the start-stop-daemon tool is used, within Debian server packages, I wonder if it might be appropriate to make something that would stand in the place of start-stop-daemon, and would integrate with runit. I only wish I would truly have the time to engage in such a venture, myself.

TJ111
TJ111

Very nice to keep a sort of hands off approach to management, no more having to manually log in and do /etc/init.d/exim restart or anything like that. Although I wonder how this handles corrupted services, ie. a bad compile or a malformed config file.

BALTHOR
BALTHOR

I think that the murder has a virus against Linux running correctly.

vdanen
vdanen

So some observation is required. =) Of course, you'd know fairly easily whether or not something was continually looping. I did write a management script called srv that you could use to start/stop/etc. services and it observed whether or not a service was looping and would put it in a down state if it was looping.

The Scummy One
The Scummy One

read like this Just interpret it a bit differently than stated "Linux should be just like Microsoft I think that the murder has a virus against Linux running correctly." He is stating that MS tries to keep Linux down, so Linux should mess with MS to cause issues with Win :D

seanferd
seanferd

No way should Linux be just like MS. Aside from the fact that you can run administrative scripts on either...

The Scummy One
The Scummy One

bringing a topic into the light, and confusing people until they 'must read' the article. Either that or THE BALTHOR is seriously 'In Touch' with the cosmos from an altered state of mind.... :D

seanferd
seanferd

running a virtual microwave inside the BIOS of my microwave oven, but by no means should Linux be like MS! I'm off to fight against this in the anti-1337 counter-torrent! Later, dudes!

OnTheRopes
OnTheRopes

I've read the articles to see if I could figure out what BALTHOR is talking about. So far I haven't.