Skip to content

Config layering and precedence

The layering is usually described as one hierarchy — deb, then overlay, then something host-local. It is not one hierarchy. Three separate questions are answered by three different pieces of code, and they do not agree on which layer wins:

  1. Resolution — which file on disk backs the name foo? Repo overlay wins.
  2. Membership — is foo switched on for agent a? Nothing wins; the layers merge as additive and subtractive deltas.
  3. Ownership — what about files the agent wrote in its own home? They are not in the contest at all, except for mini-apps, where the agent's copy wins over both.

Getting these confused is how "the overlay overrides the deb" becomes a wrong prediction about whether a newly-shipped default reaches an existing agent (it does, see Membership).

The roots on disk

Root Holds Written by In git
/usr/share/agentctl/ the shipped catalog: skills/ agents/ hooks/ mcp/ tools/ cron/ the .deb — replaced wholesale on apt upgrade no
/var/lib/agentctl/catalog, /var/lib/agentctl/tools MCP and CLI-tool catalog items added after install mcp catalog add / tool wrap-mcp — but only on a host with no repo, or with an explicit --into; the default target is the overlay no
/opt/agentctl/ (repoDir()) the overlay dirs skills/ agents/ hooks/ mcp/ tools/ apps/ cron/, plus defaults.yaml, hooks/registry.yaml and the ledger shards the CLI, auto-committed yes
~/.claude/{skills,agents,apps} (pi: ~/.pi/agent/{skills,apps}) whatever the agent authored for itself the agent no

Mini-app bundles have no payload dir: the first-party ones are embedded in the agentctl binary (appsFS), and the overlay dir is <repo>/apps/.

Resolution: overlay wins, at whole-item granularity

Every category resolves the same shape — take the repo overlay's copy if the path exists, otherwise the shipped one. For skills and subagents that is overlayPath() in internal/cli/repo.go and catalogTarget() in internal/converge/manifest.go:

func catalogTarget(o options, kind, rel string) string {
    if o.repoDir != "" {
        p := filepath.Join(o.repoDir, kind, rel)
        if _, err := os.Stat(p); err == nil {
            return p
        }
    }
    return filepath.Join(o.share, kind, rel)
}

This is a full override of the whole item, never a merge of its contents. An overlay skills/browser/ replaces the shipped skills/browser/ entirely — a file the shipped version had and the overlay does not is simply gone, because only one directory is ever reachable. The same holds for a subagent .md, an mcp.yaml item dir and a tool.yaml item dir.

Three deviations from that shape are worth knowing:

  • MCPs and CLI tools have a third root in the middle. catalogItemDir() (internal/cli/manifest.go) and toolCatalogBases() (internal/cli/tool.go) search <repo>/mcp → /var/lib/agentctl/catalog → /usr/share/agentctl/mcp. The runtime catalog is genuinely host-local — outside both the deb and the repo — and it exists so an item added after install survives apt upgrade, which overwrites the payload. It still loses to the overlay.
  • Hooks merge per key, not per file. There is one registry file per layer (<share>/hooks/registry.yaml, then <repo>/hooks/registry.yaml) and loadHookRegistry() reads both into one map, later writes winning. So an overlay registry that defines one hook overrides that hook and leaves the other shipped hooks intact. Within a key the entry is still replaced whole.
  • Per-user cron templates are not overlayable. discoverCronJobs() globs ShareDir + "/cron/units" only, so a new per-agent cron job needs its unit templates in the payload. Only the descriptors are overlayable: hostCronDir() prefers <repo>/cron/<label>/ over <share>/cron/host/<label>/, and <repo>/cron/users/<agent>/ holds self-scope descriptors.

The cron descriptor override is also the one place the codebase warns about a shadow, for a reason worth repeating. cron add --host <label> writes an overlay dir containing a copy of the script, and cron remove deliberately leaves it behind. If a label of the same name later ships as a default, the host keeps running the frozen copy and every later fix to the shipped script silently stops arriving. hostCronShadowsShipped() prints a line on apply so this stops being invisible; it does not take the overlay away, because an overlay may be a deliberate local override and apply is not the place to decide that.

Membership: a merge of deltas

Whether an item is on for an agent is a different computation, in internal/cli/catalogdefaults.go, and there is no override anywhere in it:

effective = ( deb_defaults ∪ overlay.add − overlay.remove ) + agent.add − agent.remove
  • deb_defaults — the compiled-in sets defaultSkills, defaultSubagents, defaultCron, defaultHooks, defaultTools, and the channel-conditional defaultClitoolsForChannel.
  • overlay.* — <repo>/defaults.yaml, per category, add: and remove: inline lists. Edited with agentctl defaults add|remove <category> <item>.
  • agent.* — the activate: / deactivate: lists in agents.d/<name>.yaml, written by agents skills|subagents|cron|hooks add|remove, by tool add|remove for pi clitools, and by mcp add|remove for the claude tools leg.

Both layers apply through the same primitive, applyDelta(base, add, remove) = (base ∪ add) − remove. The layered categories are skills, subagents, cron, hooks, clitools and tools.

The ledger records only the deltas, never the resolved set. That is the point of the design: a skill added to defaultSkills in a new deb reaches every existing agent on the next apply, instead of being frozen out by a recorded list captured before it existed. On every apply, reconcileAgentDeltas() recomputes the effective set from whatever the shard holds and rewrites the smallest delta that reproduces it (minimalDelta: add = eff − def, remove = def − eff). A legacy fully-resolved activate: list therefore shrinks to the agent's genuine divergence, and collapses to nothing at all once a host blesses the extra through defaults.yaml. The pass is idempotent, and it never grows a shard.

The same pass self-heals: healLeg() drops delta items that no longer name a catalog entry, so a deleted or renamed skill strips itself off every agent fleet-wide. It is gated on catalogPresent() so a dev box with no catalog installed never prunes deltas it cannot verify, and the enumerators it checks against union the overlay, so an overlay-only item counts as present.

Three scoping notes:

  • defaults.yaml has no host key. It sits at the repo root and applies to every agent resolved through that repo, which on a repo shared by several hosts means more than one host.
  • The tools (claude MCP) leg is skipped for pi agents and the clitools leg for claude agents, so a category is never materialized onto the wrong runtime.
  • agentctl defaults add tools <mcp> does not work today, even though tools is a delta category and the reader accepts a tools: block. defaultsCatalogEntries() has no tools case, so item validation always rejects it. Blessing a default MCP for a host currently means hand-editing defaults.yaml.

Ownership: the agent's own files

A skill the agent writes in ~/.claude/skills/<name>/ is not a catalog item and is not part of either mechanism above. The reconcile only manages symlinks, and it prunes conservatively (reconcileSymlinks in internal/converge/reconcile.go): inside the managed dirs it removes an entry only if it is a symlink and its target points into one of Manifest.CatalogRoots (the overlay root and the payload root) and it is not desired. A plain directory, a plain file, or a symlink pointing anywhere else is left alone. A personal one-off is safe; promote it to a shared item by dropping it in the overlay.

Name collision between the two is the sharp edge, and it fails rather than resolving. If the agent owns ~/.claude/skills/foo/ as a real directory and foo is also an activated catalog skill, the reconcile tries os.Remove on the link path and then os.Symlink. Verified behaviour: a non-empty directory survives the Remove, the Symlink returns EEXIST, and the whole reconcile errors out — the activation never lands and nothing repairs it. An empty directory is removed and replaced by the symlink silently.

Mini-apps are the exception to the whole section, and the one case where the informal "host-local wins" reading is correct. loadAppManifests() merges embedded then overlay (overlay wins on id), and mergeUserApps() then merges the agent's own ~/.claude/apps over that result — user wins on id. So an agent can shadow a shipped cockpit panel with its own, and nothing on the launcher says so. What it cannot do is claim first-party status: the reader stamps Source: "user" rather than reading it from manifest.json, and an admin: true in a user manifest is discarded.

Worked example: a skill overridden in the overlay

Agent testbot on a host whose repo is /opt/agentctl. demo-skill and repo-skill both ship in the deb; repo-skill also exists in the overlay. agentctl apply runs:

  1. reconcileAgentDeltas("testbot") re-minimizes the shard's deltas against the current effective defaults.
  2. convergeAgent (internal/cli/converge.go) calls effectiveList("testbot", "skills"), which resolves ( defaultSkills ∪ defaults.yaml add − remove ) + activate − deactivate → [demo-skill, repo-skill, …]. Note this step is pure membership: the two skills' locations have not been consulted yet.
  3. That list is handed to converge.BuildManifest as WithLists(...), along with WithPayloadShare(ShareDir) and WithRepoDir(repoDir()).
  4. buildSymlinks calls catalogTarget per item. demo-skill is absent from <repo>/skills/, so it resolves to /usr/share/agentctl/skills/demo-skill. repo-skill is present, so it resolves to /opt/agentctl/skills/repo-skill — the shipped copy at /usr/share/agentctl/skills/repo-skill is never referenced again.
  5. reconcileSymlinks points /home/testbot/.claude/skills/{demo-skill,repo-skill} at those two targets and prunes any other catalog-owned symlink in that dir.

TestBuildManifestFixture in internal/converge/manifest_test.go pins exactly this, against fixtures in internal/converge/testdata/ that carry repo-skill in both payload/skills/ and repo/skills/.

The MCP path differs only in step 4: mcp add resolves the item through catalogItemDir() (overlay → runtime catalog → payload) and renders a socket unit plus an entry in the agent's ~/.claude.json. Because that file is read only at process start, the first apply after a change to the default MCP set restarts every claude agent on the host.

Verbs

agentctl defaults list                          # per category: deb set, effective set, overlay legs
agentctl defaults add <category> <item>         # write the overlay add leg (root; commits, does not apply)
agentctl defaults remove <category> <item>      # write the overlay remove leg

agentctl agents skills add|remove <skill> <agent>|*     # the per-agent delta
agentctl agents subagents|cron|hooks add|remove ...     # same shape per category
agentctl tool add|remove <tool> <agent>|self            # the pi clitools delta
agentctl mcp add|remove <mcp> <agent>|self              # the claude tools delta

agentctl agents skills catalog add <name> [--from <dir>]   # AUTHOR into the overlay
agentctl mcp catalog add <name> --npm <pkg@ver>            # lands in the overlay by default

agentctl apply                                  # recompute, re-minimize, converge