The service watcher
Every server keeps an eye on its own services. When the web server, the database, the mail server or another service that belongs to the server's roles stops, the server starts it again by itself within a few minutes — without the panel, wi
Written for: Administrator
Every server keeps an eye on its own services. When the web server, the database, the mail server or another service that belongs to the server's roles stops, the server starts it again by itself within a few minutes — without the panel, without the network and without anybody having to do anything. That is the service watcher. It is on by default, on every server.
What the service list shows
On a server's Configuration page every service has a Watch column:
| What it says | What it means |
|---|---|
| on | The server restarts this service when it stops. |
| always watched | The agent, ssh and the firewall. These cannot be excluded or paused; the reason is shown below. |
| excluded | You excluded this service. It is not restarted, but you are still told when it is down. |
| stopped by … at … | Somebody stopped the service through CoreCP. The watcher leaves it alone until it is started again, and it raises no outage alert. |
| paused until … | The watcher is paused. After that time it resumes by itself. |
| given up since … | The service kept stopping. After three attempts within fifteen minutes the server stops trying; you get an alert with the last log lines. After an hour, or as soon as you start the service yourself, it tries again. |
| off on this server | The watcher is off for this whole server. |
| unknown | The panel cannot read the server right now. It never says "on" then. |
Below the state, last restarted automatically at … shows when the watcher last restarted the service, if it ever did.
Excluding a service
If you do not want the server to restart a service by itself — because you are not using it for a while, say — click Exclude on that service. Watch again undoes it. An excluded service stays in the list and still raises the "service down" alert: excluding means "do not touch it", not "do not tell me".
Pausing the watcher
If you are about to work on the server yourself and do not want services restarted in between, click Pause the watcher above the service list and choose how long: 15 minutes, 1 hour, 4 hours or 24 hours. Longer is not possible, and a pause without an end does not exist: it always ends by itself. While the pause runs a bar above the list offers Resume now. The agent, ssh and the firewall stay watched during a pause.
Turning the watcher off
At the bottom of the page, with the buttons that affect the machine, is Turn the watcher off. That is permanent: the watcher stays off until somebody turns it on again, which is why the panel asks you to type the server's name. For maintenance, a pause is almost always the better choice.
Notifications
- A service is down — while the watcher is still restarting it this is a warning. When the watcher gives up it becomes critical, and the message says how often the server tried and what the service last wrote to its log. A service stopped deliberately through CoreCP does not raise it.
- A service was restarted automatically — a light notice: it is in the bell and nowhere else. If you also want it by mail or as a push notification, switch that on under Settings → Notifications → Light notices.
Who may do this
Excluding, pausing, resuming and turning the watcher on or off is for an administrator or a server administrator of the server — the same people who may start and stop services. Resellers and customers do not see the service list.
From the terminal
corectl node service watch status
corectl node service watch pause --for 1h
corectl node service watch resume
corectl node service watch exclude proftpd
corectl node service watch include proftpd
corectl node service watch off --confirm server.example.com
corectl node service watch on