Following background tasks

Not everything is instant. Creating a website takes seconds, requesting a certificate half a minute, restoring a backup sometimes an hour, and migrating a whole server a night. All of that work runs on the server, not in your browser: you may close the tab, shut the laptop and come back tomorrow.

This page is about where to find that work, what the states mean, and why some tasks show a counter and most do not.

1. Where to find it

In two places, and they answer slightly different questions.

The clock in the top bar, to the left of the bell. The number on it is how many things are running; a small red dot below it means something failed recently. Press it and you see what is running, with the last ten finished ones underneath — each with its duration and, where there is one, its counter. On a phone the same list slides up from the bottom.

This is also where a task "moves to". Start something and you see it right under the button you pressed; that block goes away when you click on, but the clock holds on to it. So you may navigate away. That is exactly what used to go wrong: you started a restore, went to look at your mail, and the panel had forgotten anything was running. The server had not.

The Tasks page (menu: Insight › Tasks) is the whole list, with filters, and where you go to find something from last week. Above the table are two buttons: Everything and Needs attention. The second shows only what failed or whose state became unknown — the two things you have to do something about. Beside them you filter by state, origin, server and period, and the search box searches the operation, the subject and the server name.

Tasks or Notifications? Two windows onto the same events. Tasks is the state of play: what is running, how far along, and what came out of it. Notifications (the bell) is the history of what you were told, with read/unread and an archive. A task that finishes produces exactly one notification — never two.

2. What is in the list

One list for everything running in the background, from four kinds of origin:

Every row tells the same story: when it started, what it is, what it is about, which machine, who asked for it, how long it is taking and how it stands.

You only ever see your own work. Exactly the same boundary as the audit log and your notifications: as a customer you see what you asked for, as a reseller what you and your customers asked for, as an administrator everything. That is not a separate rule for this list but the same rule, written once — so a task can never appear here that the audit log would not show you.

3. The states

• Queued — accepted, but the machine is busy with something else. A server applies its changes one at a time so that two operations can never get in each other's way.
• Running — it is going.
• Done — it worked.
• Failed — it did not, with the reason. You get one of those as a notification too, so you do not have to sit and watch.
• Cut short — the task had a time limit and reached it, or the server was restarted underneath it. This is emphatically not failed: nothing is broken, the work simply was not finished. What was already done stays done, and the log shows how far it got. This state does count towards needs attention.
• Stopped — somebody pressed stop. That is a choice rather than a problem, so it does not ask for attention.
• Unknown — this is the honest state. The panel has been unable to reach the server for five minutes (or the server no longer has the task), so it does not know whether the work succeeded. The row says since when it went quiet. A reboot or a brief network problem takes less than five minutes, and a task stays "running" straight through those.

4. Why not everything has a bar

Some tasks carry a counter — 3 of 11 accounts — and most do not. That is a choice, not an unfinished feature.

A bar may only appear when something was really counted. An operation that built a list before it started knows how many items are in it and can honestly say how far along it is. Today that is three of them:

• a backup of every account on a server: accounts;
• a migration of a whole server: accounts;
• a bulk action across your WordPress sites: sites.

Everything else shows the state and the elapsed time, and no percentage. The reason is that all the alternatives lie: a bar based on elapsed time sits at 80% while there is still half an hour to go, and a bar based on how many log lines have arrived speeds up precisely when something goes wrong and a lot gets written. Better no number than a wrong number.

A counter also never runs backwards — not even for a migration doing ten accounts at once and reporting them in whatever order they finish.

5. Reading the output back

Press a row — in the list or in the clock — and a panel slides open with that operation's full log, with the warnings lifted out and marked at the top. The same log you watch scroll past under the button while it runs, but afterwards and complete.

If the task failed, the reason is at the top in red, in the server's own words — not "something went wrong" but, for instance, "the address in DNS points at 203.0.113.9 and this server listens on 185.117.226.120". At the bottom there is one button back to what the task was about: the account, the domain, the rollout or the migration.

An opened panel has an address of its own (/tasks?task=…), so you can send the link to a colleague or to support.

Two retention periods, and the difference is visible:

So a task from two months ago is still there, but says "the log expired on the server" rather than showing a blank page. That is deliberate: blank would look like a fault.

A rollout or a migration carries no log here but a link to its own page, because that is where the full story per server or per account lives.

6. Stopping — and what stopping is not

Open a task that is still going and there is a Stop button. There was not one for a long time, and for a good reason: while the server itself had no "cancel", a stop button would only have looked as though it did something. The server has one now, so the button is there — with exactly the same honesty.

The panel tells you what will happen before you confirm, and there are three possible outcomes:

• Stopped. The task had not started. Nothing on the server was changed.
• Asked to stop. The task was already running. It stops at the next point where the server is in a tidy state — so it may still finish. That is not hedging but the truth: breaking off a change halfway would leave the server in a state nobody designed.
• Too late. It had already finished. Nothing changes.

Stopping is not a rollback. What was already done stays done; the log under the task shows how far it got. If you want something undone, do that afterwards with the ordinary operations — most are reversible, and the ones that are not ask you to confirm first.

A rollout and a migration are stopped on their own pages, not here. Two stop buttons for the same work would sooner or later do something different.

6a. "The server is busy" on webmail or phpMyAdmin

Press Webmail or phpMyAdmin while the server happens to be applying another change and you get an amber message within a second: the server is busy with another change, so signing in did not start; nothing was changed. Try again in a moment — usually it is finished by then.

That is a deliberately fast refusal rather than a waiting screen. Signing in is also a change on the server, so it used to stand in the same queue and you watched "signing you in…" for up to a minute before getting the same answer. The same answer, a minute earlier, is the whole difference.

7. For administrators: from the command line

# Everything running right now, across the fleet.
root@panel1:~# corecp-panel tasks list --active

# And what was cut short on the way.
root@panel1:~# corecp-panel tasks list --state interrupted --limit 10
STARTED              SOURCE  OPERATION        SUBJECT  NODE              STATE    PROGRESS       DURATION  ID
2026-08-17 09:41:02  node    backup.accounts  -        stck1.corecp.dev  running  3/11 accounts  1m48s     6f1e…

# Only today's failures.
root@panel1:~# corecp-panel tasks list --state failed --limit 10

# One task whole, including the reason behind its state.
root@panel1:~# corecp-panel tasks show 6f1e2c40-0000-4000-8000-000000000001

# Bring the panel up to date without waiting for the loop (useful after an outage).
root@panel1:~# corecp-panel tasks reconcile
4 row(s) reconciled or projected

And over the API, with an API key that may read the tasks family:

$ curl -s -H "Authorization: Bearer $KEY" \
    'https://panel1.corecp.dev/api/v1/tasks?active=1&limit=5'
$ curl -s -H "Authorization: Bearer $KEY" \
    'https://panel1.corecp.dev/api/v1/tasks/6f1e2c40-…'

The technical side is in docs/tasks.md.