Config layering and precedence

The layering is usually described as one hierarchy — deb, then overlay, then something host-local. It is not one hierarchy. Three separate questions are answered by three different pieces of code, and they do not agree on which layer wins:

1. Resolution — which file on disk backs the name foo? Repo overlay wins.
2. Membership — is foo switched on for agent a? Nothing wins; the layers merge as additive and subtractive deltas.
3. Ownership — what about files the agent wrote in its own home? They are not in the contest at all, except for mini-apps, where the agent's copy wins over both.

Getting these confused is how "the overlay overrides the deb" becomes a wrong prediction about whether a newly-shipped default reaches an existing agent (it does, see Membership).

The roots on disk

Root	Holds	Written by	In git
/usr/share/agentctl/	the shipped catalog: skills/ agents/ hooks/ mcp/ tools/ cron/	the .deb — replaced wholesale on apt upgrade	no
/var/lib/agentctl/catalog, /var/lib/agentctl/tools	MCP and CLI-tool catalog items added after install	mcp catalog add, but only on a host with no repo or with an explicit --into; the default target is the overlay. tool catalog add always writes the overlay	no
/opt/agentctl/	the overlay dirs skills/ agents/ hooks/ mcp/ tools/ apps/ cron/, plus defaults.yaml, hooks/registry.yaml and the ledger shards	the CLI, auto-committed	yes
~/.claude/{skills,agents,apps} (pi: ~/.pi/agent/{skills,apps})	whatever the agent authored for itself	the agent	no

Mini-app bundles have no payload dir: the first-party ones are embedded in the agentctl binary, and the overlay dir is <repo>/apps/.

Resolution: overlay wins, at whole-item granularity

Every category resolves the same shape — take the repo overlay's copy if the path exists, otherwise the shipped one. That applies equally to a skill, a subagent, an MCP catalog item and a CLI tool item.

This is a full override of the whole item, never a merge of its contents. An overlay skills/browser/ replaces the shipped skills/browser/ entirely — a file the shipped version had and the overlay does not is simply gone, because only one directory is ever reachable. The same holds for a subagent .md, an mcp.yaml item dir and a tool.yaml item dir.

Three deviations from that shape are worth knowing:

• MCPs and CLI tools have a third root in the middle. Resolution searches <repo>/mcp → /var/lib/agentctl/catalog → /usr/share/agentctl/mcp. The runtime catalog is genuinely host-local — outside both the deb and the repo — and it exists so an item added after install survives apt upgrade, which overwrites the payload. It still loses to the overlay.
• Hooks merge per key, not per file. There is one registry file per layer (<share>/hooks/registry.yaml, then <repo>/hooks/registry.yaml), and both are read into one combined map, with the overlay's entries winning per key. So an overlay registry that defines one hook overrides that hook and leaves the other shipped hooks intact. Within a key the entry is still replaced whole.
• Per-user cron templates are not overlayable. Cron unit templates are discovered only from the shipped payload's cron/units directory, so a new per-agent cron job needs its unit templates in the payload. Only the descriptors are overlayable: a host-level cron descriptor prefers <repo>/cron/<label>/ over <share>/cron/host/<label>/, and <repo>/cron/users/<agent>/ holds self-scope descriptors.

The cron descriptor override is also the one place the tool warns about a shadow, for a reason worth repeating. cron add --host <label> writes an overlay dir containing a copy of the script, and cron remove deliberately leaves it behind. If a label of the same name later ships as a default, the host keeps running the frozen copy and every later fix to the shipped script silently stops arriving. agentctl apply prints a line calling this out so it stops being invisible; it does not take the overlay away, because an overlay may be a deliberate local override and apply is not the place to decide that.

Membership: a merge of deltas

Whether an item is on for an agent is a different computation, and there is no override anywhere in it:

effective = ( deb_defaults ∪ overlay.add − overlay.remove ) + agent.add − agent.remove

• deb_defaults — the compiled-in default sets for skills, subagents, cron jobs and hooks, and the default MCPs, which are channel-conditional on the tools leg (a Teams agent is handed the teams MCP, never the telegram one).
• overlay.* — <repo>/defaults.yaml, per category, add: and remove: inline lists. Edited with agentctl defaults add|remove <category> <item>.
• agent.* — the activate: / deactivate: lists in agents.d/<name>.yaml, written by agents skills|subagents|cron|hooks add|remove, and by mcp add|remove for the tools leg — the same verb against the same key on both runtimes.

Both layers apply through the same additive-then-subtractive rule: union in the adds, then remove the removes. The layered categories are skills, subagents, cron, hooks and tools.

The ledger records only the deltas, never the resolved set. That is the point of the design: a skill added to the compiled-in defaults in a new deb reaches every existing agent on the next apply, instead of being frozen out by a recorded list captured before it existed. On every apply, the effective set is recomputed from whatever the shard holds, and the smallest delta that reproduces it is written back (add = effective − default, remove = default − effective). A legacy fully-resolved activate: list therefore shrinks to the agent's genuine divergence, and collapses to nothing at all once a host blesses the extra through defaults.yaml. The pass is idempotent, and it never grows a shard.

The same pass self-heals: it drops delta items that no longer name a catalog entry, so a deleted or renamed skill strips itself off every agent fleet-wide. That pruning is gated on the catalog actually being present, so a dev box with no catalog installed never prunes deltas it cannot verify, and the check unions the overlay, so an overlay-only item counts as present.

Three scoping notes:

• defaults.yaml has no host key. It sits at the repo root and applies to every agent resolved through that repo, which on a repo shared by several hosts means more than one host.
• No category is skipped by runtime. Every layered category reconciles for every agent, on either runtime: there is no wrong runtime for a tools: delta to be materialized onto, because a pi agent wires the same catalog MCP over the same socket a claude agent does.
• agentctl defaults add tools <mcp> blesses a default MCP for every agent on a host — it resolves against the MCP catalog, so tools validates like any other delta category:

agentctl defaults add tools <mcp>                # bless a default MCP for this host

Ownership: the agent's own files

A skill the agent writes in ~/.claude/skills/<name>/ is not a catalog item and is not part of either mechanism above. The reconcile only manages symlinks, and it prunes conservatively: inside the managed dirs it removes an entry only if it is a symlink and its target points into one of the catalog roots (the overlay root and the payload root) and it is not desired. A plain directory, a plain file, or a symlink pointing anywhere else is left alone. A personal one-off is safe; promote it to a shared item by dropping it in the overlay.

Name collision between the two is the sharp edge, and it fails rather than resolving. If the agent owns ~/.claude/skills/foo/ as a real directory and foo is also an activated catalog skill, the reconcile tries to remove the existing path and then symlink the catalog target in its place. Verified behaviour: a non-empty directory survives the removal attempt, the symlink step then fails because the path still exists, and the whole reconcile errors out — the activation never lands and nothing repairs it. An empty directory is removed and replaced by the symlink silently.

Mini-apps are the exception to the whole section, and the one case where the informal "host-local wins" reading is correct. App manifests are merged embedded then overlay (overlay wins on id), and the agent's own ~/.claude/apps is then merged over that result — user wins on id. So an agent can shadow a shipped cockpit panel with its own, and nothing on the launcher says so. What it cannot do is claim first-party status: the reader always stamps a user app's source as "user" rather than reading it from manifest.json, and an admin: true or supervisor: true in a user manifest is discarded.

Worked example: a skill overridden in the overlay

Agent testbot on a host whose repo is /opt/agentctl. demo-skill and repo-skill both ship in the deb; repo-skill also exists in the overlay. agentctl apply runs:

1. The shard's deltas are re-minimized against the current effective defaults.
2. Membership resolves for testbot's skills category: ( deb_defaults ∪ defaults.yaml add − remove ) + activate − deactivate → [demo-skill, repo-skill, …]. Note this step is pure membership: the two skills' locations have not been consulted yet.
3. That list feeds resolution, which decides where each item actually lives on disk.
4. demo-skill is absent from <repo>/skills/, so it resolves to /usr/share/agentctl/skills/demo-skill. repo-skill is present, so it resolves to /opt/agentctl/skills/repo-skill — the shipped copy at /usr/share/agentctl/skills/repo-skill is never referenced again.
5. /home/testbot/.claude/skills/{demo-skill,repo-skill} are symlinked at those two targets, and any other catalog-owned symlink in that directory is pruned.

The MCP path differs only in the resolution step: mcp add resolves the item through the overlay → runtime catalog → payload search order, and renders a socket unit plus an entry in the agent's ~/.claude.json. Because that file is read only at process start, the first apply after a change to the default MCP set restarts every claude agent on the host.

Verbs

agentctl defaults list                          # per category: deb set, effective set, overlay legs
agentctl defaults add <category> <item>         # write the overlay add leg (root; commits, does not apply)
agentctl defaults remove <category> <item>      # write the overlay remove leg

agentctl agents skills add|remove <skill> <agent>|*     # the per-agent delta
agentctl agents subagents|cron|hooks add|remove ...     # same shape per category
agentctl mcp add|remove <mcp> <agent>|self              # the tools delta, on BOTH runtimes

agentctl agents skills catalog add <name> [--from <dir>]   # AUTHOR into the overlay
agentctl mcp catalog add <name> --npm <pkg@ver>            # lands in the overlay by default

agentctl apply                                  # recompute, re-minimize, converge