Using the CLI¶

The CLI reference lists every verb and its flags. This page is the other half: the sequences an operator runs, in the order they run them, and the places where the obvious next command is not the right one.

Where a verb runs, and why that changes what you type¶

Every verb takes one of two routes, decided by the Mutating set in internal/proto/verbs.go:

Mutating verbs — apply, agents add, agents restart, and every secret subcommand including get and list — go over the unix socket to agentd, which authorizes the caller by SO_PEERCRED and re-execs the verb as root.
Read-only verbs — status, agents list, queue, logs, mcp catalog list — run in the calling process with no daemon hop, except for the cross-agent and fleet-wide reads that need root to open another agent's home.

From a shell you need sudo, because a mutating verb's own requireRoot() runs before the socket does. From an admin agent you do not: agentd's authorize() returns allowed for an operator-scope caller and re-execs as root on its behalf, so the verb sees the root it demanded. That is the intended way to drive a fleet — see the admin role.

provision is the exception in both directions. It is the bootstrap that installs agentd, so it can never route through it, and it always runs locally as root.

Bringing a new agent online¶

agents add is not a ledger write. It provisions the Linux user, converges the agent's whole manifest (files, symlinks, units, display), renders the env file, records and commits the shard, and starts the unit — inline, in that one command. No apply follows it.

sudo agentctl agents add assistant --channel webview --user-email person@example.com
sudo agentctl agents status assistant      # unit, uptime, restarts, mem/cpu, creds, shim
sudo agentctl doctor assistant             # the same facts as problems, each with a fix line

Two things it deliberately does not do.

It does not log the agent in. A claude agent's unit starts with no Claude credentials; they arrive later, either from a self-serve OAuth flow on the agent's channel or from the host side:

sudo agentctl login claude assistant           # PKCE
sudo agentctl login claude assistant --status  # prints logged-in | needs-login

A pi agent boots on a provider API key instead. agents add --runtime pi starts the unit only if that key is already in the store; without one it enables the unit, leaves it stopped, reports NEEDS-LOGIN, and names the secret set + agents restart pair that fixes it.

It does not push. The shard is committed to /opt/agentctl and left there. On a fleet sharing one repo, run sudo git -C /opt/agentctl push afterwards or no other host learns the agent exists (Multi-host fleet).

Adding and removing skills and MCPs¶

Both take effect immediately; neither needs apply. Both also record the per-agent delta in the shard, so a later apply reproduces the same state — see Config layering § Membership.

agentctl agents skills list                          # the catalog plus the default-seeded set
sudo agentctl agents skills add browser assistant    # symlinks it in, records the delta
sudo agentctl agents skills add browser '*'          # every agent — operator/admin scope only

A skill is a directory the agent reads per turn, so the symlink is the whole change and nothing restarts.

An MCP is a process, and the agent's wiring for it is read once at start:

agentctl mcp catalog list                    # what is pinned on this host, and its auth mode
sudo agentctl mcp add ynab assistant         # installs the runtime, starts the socket unit, wires ~/.claude.json
sudo agentctl agents restart assistant
sudo agentctl mcp remove ynab assistant      # detaches; this one prints `restart to apply`

The restart is easy to skip because only half the pair asks for it: mcp remove prints restart to apply: agentctl agents restart <agent> and mcp add prints nothing of the kind, although ~/.claude.json is read only at process start either way.

Activating an MCP whose credential is missing is the most common way to turn doctor red. An activated MCP with a required_env key absent from the secret store is a FAIL, not a warning, because the agent starts and the tool silently does not.

A ledger change: `set` records, `apply` renders¶

agents set writes one scalar into agents.d/<name>.yaml, commits it, and stops. Nothing on the host has moved yet, and the command prints the one that moves it:

✓ assistant.model = <id> — run agentctl apply assistant to reconcile the runtime.

sudo agentctl agents set assistant model <id>
sudo agentctl apply assistant --dry-run    # the plan, including the exact ↻ restart set
sudo agentctl apply assistant              # converge, then restart what changed

A bare name after apply is the selector, not a path to an agents.yaml; the two are told apart by whether the argument looks like a path.

Four settable scalars break the record-then-render rule, and knowing which saves an apply or explains one you did not ask for:

scalar	what `agents set` does by itself
`channel`, `runtime`	runs `apply <name>` for you — both bake into on-disk unit names that nothing re-derives, so a stale unit would keep running the old transport indefinitely
`user_email`	reconciles the web-view gateway's email→agent ledger at once, so the new address is routable without a host-level apply
`cost_cap_usd`	writes the value into the agent's env; live on the next turn, subsession or cron tick, with no apply and no restart
everything else	records only

Secrets live in a separate store and behave differently again. A per-agent secret is rendered into the unit's EnvironmentFile and the agent restarted, by secret set itself, if that unit is currently active:

sudo agentctl secret set ANTHROPIC_API_KEY --agent assistant   # value on stdin; --value V also works

A shared secret (no --agent) does neither. Its blast radius is fleet-wide so apply owns it, and secret set does not say so — run apply yourself. Empty stdin with no --value is refused rather than written, so a mistyped pipe cannot blank a key.

Cron jobs are like skills rather than like scalars: cron add writes the overlay descriptor, records the label, and installs and starts the systemd timer inline. The one case that waits for an apply is a --host cron pinned with --run-on to a different machine, which that machine converges on its own next run. See Crons.

Reading an `apply`¶

apply streams the reconcile back over the socket as it runs. Before planning anything it pull --rebases the repo from origin when one is configured; --no-pull reconciles the checked-out tree as-is.

It is queued. agentd serializes reconciles behind one FIFO ticket. If yours is not first you get queued behind N… and then the stream. agentctl queue prints the register — ID STATE VERB CALLER RESULT — and is the first thing to check when some unrelated verb appears to hang.
Five verbs skip that queue. login, notify, inject, agent-message and ws write take a per-agent lock with a timeout instead, so reaching an owner never waits behind a fleet reconcile.
Ctrl-C cancels the run; it does not detach from it. agentd watches the client connection and cancels the reconcile's context on EOF, which kills its whole process group. That is deliberate — a hung-up client must not orphan a half-finished converge — and it is the opposite of what habit predicts.
To detach on purpose, ask for it. --async (equivalently --no-wait) is stripped client-side and tells agentd to register the job and answer pending (request <id>) without streaming.
To re-attach, agentctl apply logs. It is a control verb answered outside the FIFO, so a second terminal can tail a run it did not start without disturbing the first.

--force reconciles and restarts every agent and gateway regardless of fingerprint. A plain apply already restarts what changed, so --force is the "restart it anyway" override, not the thorough version. The remaining flags are in Operations § apply semantics.

Checking a fleet¶

There is no terminal dashboard; the live grid is the cockpit's fleet mini-app. The CLI answers one question at a time.

agentctl status              # THIS host: agents group, slices, MCP sockets, config validity
agentctl agents list         # who is on it
agentctl agents status       # per agent: unit state, uptime, NRestarts, mem/cpu, creds, shim
agentctl health              # the same sweep folded into an exit code
agentctl timers              # schedule, next fire, last run and result
agentctl creds               # per (agent, provider) login health
agentctl usage --days 7      # token and estimated-cost rollup

health exits 1 if anything FAILs and 0 otherwise; WARNs never move the exit code, which is what makes it usable as a watchdog signal. doctor exits 1 on a FAIL, 2 on an unusable agent name, and 0 on WARNs or a clean sweep — and unlike health it prints a fix: line under each finding.

Run these as root or as an admin agent. A plain agent gets only its own row, and even that is partial: credential files are 0600 inside a 0750 home, so a non-root caller is told it cannot read them rather than shown a clean bill it did not earn.

Troubleshooting a stuck agent¶

agentctl doctor assistant                       # start here
agentctl logs assistant --since '-10 min'       # the unit journal, via agentd
agentctl logs assistant -f
sudo agentctl agents restart assistant

doctor's findings separate causes that want different fixes:

finding	what it means
`unit not installed`	the ledger knows this agent and the host does not — `apply` has not run here, or the agent is pinned to another host
`unit failed/<substate>`	the process is down; read the journal first, or the restart destroys the evidence
`crash-looping (NRestarts=…)`	it fails faster than it stays up — `NRestarts ≥ 50`, or SubState `auto-restart`
`<provider> creds dead`	a login expired; `agentctl login <provider> <name>`
`activated MCP "x" is missing its required_env secret "K"`	the agent starts, that tool does not

doctor does not check the shim, and the state it therefore cannot report is the confusing one: a claude agent whose unit is active but whose harness process is gone. The cockpit still lists it and every operation is dropped. agents status is what shows it, on the shim: line, in as many words:

  shim:     DOWN — agent is up but has NO shim: the cockpit lists it and every op is dropped

If the CLI rather than the agent is what hangs, check agentctl queue before anything else. Every mutating verb waits on the same FIFO, so one wedged reconcile makes the whole tool look broken.

Taking an agent out of service¶

sudo agentctl agents remove assistant --yes

It stops the units, runs userdel -r, and prunes the agent's timers, sockets and credentials. Without --yes it refuses with exit 2 after printing what it would destroy. The home directory is wiped, not archived.

The opposite case — an agent live on this host that the ledger no longer lists, usually after pulling someone else's deletion — belongs to apply:

sudo agentctl apply --prune         # list the orphans
sudo agentctl apply --prune --yes   # tear them down

Driving it from an admin agent¶

Everything above works unchanged in an admin agent's own shell, minus the sudo. Three verbs exist mainly on that route:

agentctl inject <agent>|self <text>                          # deliver a turn into a live session
agentctl notify <agent>|self|admin <text> [--file <path>]    # outbound to the owner, over its channel
agentctl host operation run <host> apply                     # an allowlisted verb on ANOTHER fleet host

inject and notify are the two directions of talking to an agent from a script: one puts words in as though the owner had typed them, the other sends words out to the owner over whatever channel that agent has. Both are converge-exempt, so neither waits behind a reconcile.

host operation run crosses hosts through the web-view gateway rather than over SSH — the target agent dialled in, so it traverses NAT and needs no key on the source. The self-disruptive verbs (apply, host upgrade, agent and gateway restarts and installs, login, migrate) are submitted durably: it prints operation <id>, follows the coordinator's journal, and exits with the operation's real code. Killing the follow does not touch the operation — resume it with agentctl host operation logs <id> --follow.