Phase 5 — Upstream spec drift detection
IMPL-0005: Phase 5 — Upstream spec drift detection
Section titled “IMPL-0005: Phase 5 — Upstream spec drift detection”Status: Completed Author: Donald Gifford Date: 2026-09-13
All three phases landed on feat/impl-0005-spec-drift-tool (PR #59).
One Phase 1 task is deferred - human required: the three-dispatch
issue-lifecycle sequence needs the workflow on the default branch, so it
cannot run from a working copy. Everything else is done and verified.
- Objective
- Scope
- Implementation Phases
- File Changes
- Testing Plan
- Dependencies
- Resolved Decisions
- Open Questions
- References
Objective
Section titled “Objective”Build the specdrift tool, its weekly workflow, and the guardrail test
specified in
DESIGN-0006, so that
changes to the Claude Code artifact specs (skill, agent, and command
frontmatter; plugin and marketplace manifests; hooks; MCP; the built-in
tool list) surface within a week as a tracking issue and, once the code
is tied to the digest, as a failing test that names the Go identifier to
update. Along the way, close the drift that has accumulated since
INV-0006: three missing hook events, a 45-versus-18 tool list, two
undocumented marketplace source kinds, and one missing reserved name.
Implements: DESIGN-0006 (all twelve design questions resolved 2026-09-12, every one on option (a); this doc’s task references such as “per DESIGN OQ3” point at that document’s Resolved Decisions).
In Scope
Section titled “In Scope”internal/upstreamlibrary: source table, fetcher, GFM table scanner, one extractor per digest section, deterministic digest and lock emitters, structural diff with text / JSON / Markdown renderers.cmd/specdriftthinmainwithpull,digest,diff,check,render, andvalidate-fixturessubcommands.- The committed
internal/upstream/digest.json,sources.lock.json,acknowledged.json, andruntime_fixtures.json. .github/workflows/spec-drift.yml,scripts/spec-drift-issue.sh, thespec-driftlabel, and thespec-check/spec-sync/spec-validate-fixturesrecipes.- The guardrail test and the exported parser key lists it needs.
- Catch-up fixes the guardrail demands on day one:
KnownHookEvents,KnownTools, marketplacearchiveandcommandsource kinds, theclaude-tag-pluginsreserved name, a deprecated-and-removed tools table backed byartifact.DeprecatedTools, and acknowledgements for every documented field no rule consumes. - The rendered
docs/rules/upstream-spec.mdpage, theclaudelint versionspec line, and therules --jsonupstream_versionfield. - Ruleset version bump, rules-doc and README updates, one dogfood pass per code PR, CLAUDE.md updates.
Out of Scope
Section titled “Out of Scope”- New lint rules for newly documented fields (for example a
skills/portable-fieldsrule over the six-field allowlist). The digest makes them cheap to write later; they get their own IMPL. - Parsing documented fields that no rule consumes (see OQ4). Presence in the digest plus an acknowledgement is the deliverable.
- Generating
knowndata.gofrom the digest (DESIGN OQ7 chose guard-only). - Deep extraction for experimental surfaces (monitors, themes, channels, LSP): field names only.
- Committing raw upstream pages in any form (DESIGN OQ6).
- Changes to
just ci,just test, orci.ymlbeyond the onerender --checkstep in Phase 3. Everything that touches the network lives inspec-drift.ymland thespec-*recipes.
Implementation Phases
Section titled “Implementation Phases”Three PRs, one per phase, in the order DESIGN OQ10 fixed. Phase 1 changes
no shipped binary; Phases 2 and 3 do and ship as minor releases through
the label-driven flow. Every code phase follows the house checklist:
table-driven tests, just check green, 55% coverage floor per
internal/ package, just lint-md and just docs-check for any docs
touched, fingerprint guardrail acknowledged if it moves, and no bot
commits from the workflow.
Phase 1 — Tool, digest, and workflow
Section titled “Phase 1 — Tool, digest, and workflow”Land the library, the CLI, the first committed digest, and the weekly workflow that files the tracking issue. After this phase upstream changes are visible within a week; nothing in the linter changes yet.
Branch chore/spec-drift-tool; label per OQ1.
-
internal/upstream/source.go: the source table from DESIGN §1 as a sorted slice ofSource{ID, Tier, URL, Ext}(fifteen entries, Tiers A–D — this doc previously said sixteen; the DESIGN §1 table is the source of truth and lists fifteen), plus the twocode.claude.com/schemas/*.jsonprobes, which are fetched on every run but never fail one. They are recorded in the work-directory manifest with their HTTP status and kept out of the committed lock (see Amendments). -
internal/upstream/fetch.go:Fetch(ctx, sources, workDir)usinghttp.NewRequestWithContext(thenoctxandbodycloselinters are on), three retries with backoff on 5xx and transport errors, a 30-second per-source timeout, aUser-Agentnaming the repo, redirects followed. Writes<id>.<ext>andmanifest.json(url,status,sha256,etag,last_modified,bytes) into the work directory. Any Tier A–D source failing after retries is a hard error (exit 2). -
internal/upstream/table.go: line-oriented GFM table scanner (header row, delimiter row, body rows split on unescaped|, backticks stripped from the first cell) and section scoping: from an anchor heading to the next heading of equal or higher level. No Markdown dependency added. -
internal/upstream/digest.go: theDigest,Field, and section types from DESIGN “Data Model”; aMarshalthat emits sorted keys, sorted arrays, two-space indent, LF, trailing newline;digest_versionpinned at 1. -
Extractors, one file per source, each declaring source id, anchor regex, columns, sanity floor (per OQ8), and implementing the
Extractorinterface with hard failure on a missing anchor, unexpected headers, or an under-floor row count (DESIGN OQ11):-
extract_skills.go—skills.frontmatter(### Frontmatter reference),skills.portable_fields(#### Using skill frontmatter outside Claude Code),skills.substitutions(#### Available string substitutions). -
extract_agents.go—agents.frontmatter(first table under### Write subagent fileswhose first row isname) andagents.enumsfrom the backticked tokens in themodel,permissionMode,effort,memory,isolation, andcolordescription cells. -
extract_plugins.go—plugins.manifest_fields(the three tables under## Plugin manifest schema),plugins.locations(### File locations reference),plugins.substitution_fields(### Environment variables). -
extract_marketplaces.go—marketplace.fields,marketplace.owner_fields,marketplace.plugin_entry_fields,marketplace.sources(one table per###under## Plugin sources;archiveandcommandincluded), andmarketplace.reserved_namesfrom the**Reserved names**paragraph. -
extract_hooks.go—hooks.events(every###under## Hook events),hooks.types,hooks.handler_fields(per-type tables under### Hook handler fields),hooks.timeout_defaults(numbers parsed from thetimeoutrow: by type and by event override). -
extract_mcp.go—mcp.transportsfrom themcp.mdtypeenumeration;mcp.server_fieldsfrom the SchemaStore plugin-manifestmcpServersdefinition (DESIGN OQ5), cross-checked against theplugins-reference.mdMCP section. -
extract_tools.go—tools.builtinfrom the first table after thetools-reference.mdH1; each row also carriesdeprecated: truewhen its description cell begins with “Deprecated” (the docs’ own signal, consumed by the Phase 2DeprecatedToolsguardrail row). -
extract_schemastore.go—propertieskeys and the hook-event, hook-type, MCP-type, source-kind, anddefaultModeenums from the three schemas via a small JSON-pointer walk. -
extract_portable.go—portable.*from the agentskills spec table, theMAX_*constants andALLOWED_FIELDSinvalidator.py, and the allowlist, name regex, and description constraints inquick_validate.py. -
extract_changelog.go—meta.claude_code_latestfrom the first## X.Y.Zheading;meta.docs_max_markeras the highestv2.N.Nmarker across the docs pages (also written per page into the lock asversion_marker).
-
-
disagreements.go: compute docs-versus-SchemaStore set differences for hook events, hook types, MCP transports, marketplace source kinds, and plugin manifest properties; write them underdigest.disagreements(informational per DESIGN OQ9). -
lock.go:sources.lock.jsonwithurl,sha256,last_modified,version_markerper source. No timestamps. -
diff.go: structural diff per DESIGN §5 (Change{Path, Kind, Item, Old, New}), withmeta.*,schemastore.*, anddisagreementspartitioned into an informational block that never affects the exit code; renderers fortext,json, andmarkdown(Markdown groups by section, one line per change, and appends “sources changed without affecting the digest” from the lock diff). -
internal/upstream/command.go:NewRootCommand()(cobra, already a dependency) wiringpull,digest,diff, andcheckwith the flags from DESIGN §2;check --work DIRkeeps the fetched pages for artifact upload;check --updaterewrites the committed digest and lock; exit codes 0 / 1 / 2.cmd/specdrift/main.goonly calls it, mirroringcmd/claudelint. -
Golden snippet fixtures under
internal/upstream/testdata/snippets/produced per OQ6, one per extractor, plus one SchemaStore excerpt per schema and the three Tier C files. -
Tests: table scanner (escaped pipes, alignment rows, trailing whitespace, nested backticks, table ending at a heading versus a blank line); every extractor against its snippet with the exact expected set; negatives for missing anchor, wrong headers, and under-floor count; digest determinism (twice → identical bytes); diff goldens for add / remove / change / no-op / informational-only; fetch against
httptest.Servercovering retry, timeout, redirect, and manifest fields;checkend-to-end against anhttptest-served fixture set asserting each exit code. No test touches the network. -
Run
go run ./cmd/specdrift check --updateon the branch and commit the firstdigest.jsonandsources.lock.json; run it twice and assert a cleangit diff. -
justfile:spec-checkandspec-syncunder[group('spec')], documented as network recipes; neither joinsciorcheck. -
.github/workflows/spec-drift.ymlper DESIGN §6: weekly Monday 06:00 UTC cron,workflow_dispatch,pull_requestoninternal/upstream/**,cmd/specdrift/**, and the workflow file;contents: readat the top,issues: writeon the job; steps for check, artifact upload (90 days), step summary, PR warning, issue script, and fail-on-exit-2. -
scripts/spec-drift-issue.sh: ensure thespec-driftlabel, locate the single open issue, read the<!-- specdrift:diff-sha256 -->marker, and create / comment / close per DESIGN OQ3; a--dry-runflag prints theghcommands instead of running them. Addspec-drifttoLABEL_COLORSandLABEL_DESCRIPTIONSinscripts/labels.shand create it with that script. -
just lint-actionsandjust lint-configclean on the new workflow. -
deferred - human required — Validate on the branch before merge: a
workflow_dispatchrun against the committed digest reports no drift and creates no issue; a scratch commit that removesPreModelSwitchfrom the committed digest makes the next dispatch open an issue whose body listshooks.events added: PreModelSwitch; reverting the scratch commit makes the following run close it. Delete the scratch issue afterwards.GitHub only offers `workflow_dispatch` for workflows that exist onthe default branch, so this sequence cannot run until the PRmerges. What can be checked before merge has been: thepull-request trigger exercises the check step against live pageson this PR; `scripts/spec-drift-issue.sh --dry-run` was runagainst a real clean report (no-op) and a real drift report(creates the issue, body carries the marker and lists`hooks.events added: PreModelSwitch`); `actionlint` and`shellcheck` are clean; and the `spec-drift` label was createdwith `scripts/labels.sh`. Run the three dispatches immediatelyafter merge.Phase 3 added a second half to the same job. Its steps havealready been exercised by the pull-request trigger (run34997855440: Node installed, CLI installed, six fixtures agreedagainst v2.1.272), so what the post-merge dispatch adds is onlythe issue lifecycle. Theruntime half carries `continue-on-error` at every step, so afailure there shows up as a "did not run" section in the reportrather than as a red job — check the report, not just the jobcolour. This is the only item in IMPL-0005 that cannot becompleted from a working copy. -
CLAUDE.md: add
just spec-check/just spec-syncto Common commands and a “Upstream spec drift” bullet under Git / PR conventions describing the workflow, the label, and the rule that the digest is regenerated withjust spec-syncin the same PR as any code that catches up. -
Coverage:
internal/upstreamclears the 55% floor (just coverage-gate). -
PR opened with the label from OQ1;
just cigreen. (#59)
Success Criteria
Section titled “Success Criteria”go run ./cmd/specdrift checkfrom a clean checkout on merge day exits 0 and prints “no drift”; runningdigesttwice yields identical bytes.- Removing one hook event from the committed digest makes
checkexit 1 and the Markdown report namehooks.eventsand the event. - A fixture with its anchor heading removed makes
digestexit 2, name the source and anchor, and write no digest. - The three-run dispatch sequence (clean → staled → clean) produces exactly one issue, one comment, and one close.
just testpasses offline;just ciis unchanged and green;just lint-actionsis clean.internal/upstreamcoverage is at or above 55%.
Phase 2 — Guardrail and catch-up
Section titled “Phase 2 — Guardrail and catch-up”Tie the digest to the code. Export the parser key lists, add the guardrail test and the acknowledgement file, then make every fix the guardrail demands on day one. This is the phase that converts the weekly issue into a failing test.
Branch fix/upstream-drift-2026-09; label minor (the ruleset changes;
version per OQ3).
Shipped differently. Phases 1 and 2 landed on one branch,
feat/impl-0005-spec-drift-tool, under PR #59, because Phase 1 was
still open when Phase 2 started. The label moved from dont-release to
minor when the ruleset bump landed, which is the part that matters —
the branch name is cosmetic, the label drives the release.
-
internal/artifact: exportSkillFrontmatterKeys,CommandFrontmatterKeys,AgentFrontmatterKeys,PluginManifestKeys, andHookEntryKeysas sorted string slices;ParseSkill,ParseCommand,ParseAgent,ParsePlugin, andParseHookread keys through them instead of string literals so the lists cannot drift from the parsers. No behaviour change; existing parser tests stay green. (HookEntryKeysis one row beyond the DESIGN §7 table; note it there.) -
internal/upstream/acknowledged.jsonand a loader that rejects unknown digest paths and empty reasons;LoadEmbedded()for the digest and the acknowledgements viago:embed. -
internal/upstream/guard_test.go: one subtest per DESIGN §7 row — hook events, hook types, tools, model aliases, the four agent enum sets, the five key lists against their digest sections, marketplace source kinds againstMarketplaceSourceKind, reserved names againstreservedMarketplaceNames(exported for the test asReservedMarketplaceNames), MCP transports against the setmcp/transport-knownuses, and timeout defaults against the valueshooks/timeout-presentcites. Failure messages follow the fingerprint test: identifier, digest path, delta, and the two ways to resolve. A stale acknowledgement (item no longer upstream) fails. -
KnownHookEvents: addDirectoryAdded,PreModelSwitch,PostModelSwitch; update the count in theknowndata.gocomment, the lifecycle table underhooks/event-name-knownindocs/rules/rules.md, and the “30 events” sentence inREADME.md. -
KnownTools: add the 31 documented tools missing today; per OQ2 keepTaskwith an acknowledgement citing the v2.1.63 rename and dropBashOutput,KillShell, andMultiEdit; update theagents/tools-knownandcommands/allowed-tools-knownprose inrules.mdand README where they enumerate tools. -
artifact.DeprecatedTools(OQ2 addendum): a map of tool name to{Status, Since, ReplacedBy, Source}with statusrenamed,removed, ordeprecated. Seed it from the Claude Code changelog where an entry exists and from the first docs marker at which the tool left the table where none does:Taskrenamed toAgent(v2.1.63, INV-0006);BashOutputremoved, useTaskOutput(v2.0.64, changelog “Unshipped … BashOutputTool”);KillShellremoved, useTaskStop, andMultiEditremoved, useEdit(no changelog entry; record the docs marker and say so);TaskOutputdeprecated, useRead(v2.1.83, changelog). The two tools-known rules consult it beforeKnownTools, so a removed or renamed tool gets “removed in v2.0.64; use TaskOutput” instead of “unknown tool”; deprecated-but-documented tools stay known and produce no diagnostic.docs/rules/rules.mdgains a “Deprecated and removed tools” table underagents/tools-known(tool, status, version, replacement, source), linked fromcommands/allowed-tools-known. Guardrail row: everyremovedorrenamedentry must be absent fromtools.builtinand everydeprecatedentry present with thedeprecatedflag; a mismatch means the table is stale. - Marketplace sources (drift found while writing this doc): add
SourceArchive({"source": "archive", "url", "sha256"}) andSourceCommand({"source": "command", "command", "timeout", "mode"}) toMarketplaceSourceKindand the parser;marketplace/plugin-source-validvalidatesurlfor archive (HTTPS only,sha256as 64 hex when present) andcommandfor command sources;marketplace/external-source-skippedgains kind-aware wording for both; rules.md per-type table extended. Fixtureok/marketplaces/archive_command/. -
reservedMarketplaceNames: addclaude-tag-plugins(17 documented today); update the count in the comment and in rules.md. - Acknowledgements for everything documented that no rule consumes
(per OQ4): the ten unparsed skill fields, the nineteen unparsed
plugin manifest fields, agent
experimental, the hook handler fields outsideHookEntryKeys(if,statusMessage,once,asyncRewake,headers,allowedEnvVars,input,model), the MCPoauth.*subfields, marketplaceallowCrossMarketplaceDependenciesOn,renamesis parsed already, plugin-entryheaders/headersHelper/relevance, and the per-event timeout overrides inhooks.timeout_defaults.by_event. Each entry carries a reason that names the rule or phase that would consume it. - Verify the remaining rows come out equal with no acknowledgement: model aliases, the four agent enum sets, hook types, MCP transports, timeout defaults by type.
- Fixtures: a hooks file using the three new events; an agent and a
skill declaring newly documented tools (
EnterPlanMode,SendMessage,TaskCreate) intools/allowed-tools; the archive-and-command marketplace; a marketplace namedclaude-tag-pluginsin the reserved-name rule test. - Ruleset version bump per OQ3; amend the
RulesetVersiondoc comment ininternal/rules/version.goso known-data changes are an explicit bump trigger;docs/rules/rules.mdheader and README rows updated. The fingerprint should not move (no rule ids, severities, options, orAppliesTochange); if it does, ack it deliberately. -
just spec-syncon the branch so the digest and lock reflect upstream on the day the fixes land; guardrail green with every acknowledgement reasoned. - Dogfood:
just self-checkand adonaldgifford/claude-skillscheckout (cdinto it first — config discovery walks up from CWD); triage every new or removed diagnostic. Expect theagents/tools-knownandcommands/allowed-tools-knownwarning count to drop. - DESIGN-0006 §7 table updated for
HookEntryKeysandDeprecatedTools; PR labelledminor;just cigreen.
Success Criteria
Section titled “Success Criteria”go test ./internal/upstream/...passes with zero unacknowledged items; every acknowledgement has a reason naming a rule or phase.- Deleting
PreModelSwitchfromKnownHookEventsfails the guardrail with a message namingartifact.KnownHookEvents,hooks.events, and the missing event; adding a bogus acknowledgement for an item that is not upstream also fails. - Doc-valid fixtures using the 33 events, the 45 tools, archive and command sources, and the 17 reserved names lint with zero false positives; the existing regression fixtures still flag.
- An agent declaring
tools: BashOutputgets a warning naming v2.0.64 andTaskOutput; one declaringTaskOutputgets none; the rules.md deprecated-tools table matchesDeprecatedToolsrow for row. claudelint versionshows the bumped ruleset;just cigreen; minor release ships via the label flow.- Dogfood clean or fully triaged.
Dogfood results (2026-09-14)
Section titled “Dogfood results (2026-09-14)”just self-check on this repo: 0 diagnostics over 3 files, unchanged.
donaldgifford/claude-skills at 32d17b4, 158 files, before and after
this branch:
| Rule | Severity | Before | After |
|---|---|---|---|
agents/name-format | warning | 7 | 7 |
commands/allowed-tools-known | error | 0 | 4 |
The expected drop in tools-known findings did not happen because there
was nothing to drop: claude-skills declared none of the 31
newly-documented tools, so the catch-up removed no false positives
there. It added four true positives instead — four
infrastructure-as-code commands declare Task, which the tools
reference dropped when it was renamed to Agent in v2.1.63.
Worth knowing before this ships: the runtime still accepts Task, so
those four commands work today, and commands/allowed-tools-known is an
error rule. A repository that is clean on ruleset v1.5.0 and declares
Task will fail on v1.6.0. That is the intended reading of OQ2 — the
rename is real and silent — but it is a breaking change for a name that
still functions, and it is the reason this phase ships as a minor bump
with the deprecated-tools table in rules.md rather than as a patch.
Downstream fix is a one-word rename; severity = "warning" on the rule
is the escape hatch for anyone not ready.
Rules cannot lower the severity of an individual diagnostic: the engine
assigns severity per rule (runner.go), so “renamed tool” cannot be a
warning inside an error rule without an engine change. Not attempted
here — it would contradict DESIGN-0001 without an amendment.
Phase 3 — Runtime validator and rendered page
Section titled “Phase 3 — Runtime validator and rendered page”Put the Claude Code runtime’s own validator in the loop, give humans a page to link to, and expose the digest version from the binary.
Branch chore/spec-drift-runtime; label minor (the version and
rules --json output change).
-
internal/upstream/runtime_fixtures.jsonwith entries{path, kind, expect}per OQ5, covering at least: the two existing.claude-pluginfixtures aspass, one plugin manifest missingnameasfail, one marketplace with duplicate plugin names asfail, oneagents/directory aspass, and oneskills/directory as the probe that assertscontentsis still empty. -
specdrift validate-fixtures --claude <bin>: runsclaude plugin validate --strict --json <path>per entry with an emptyCLAUDE_CONFIG_DIR, parses the JSON, comparessuccesstoexpect, recordsclaude --version, and reports disagreements with the runtime’serrorsandwarningsverbatim (Markdown and JSON). Exit 0 / 1 / 2. Tested with a fakeclaudescript onPATHreturning canned JSON. - Workflow wiring per OQ10: Node from
mise.tomlviajdx/mise-action,npm install -g @anthropic-ai/claude-code@latest(DESIGN OQ8), runvalidate-fixtures, append its report to the drift report before the issue script runs. A runtime-step failure must not suppress the drift report. -
just spec-validate-fixturesrecipe (requires a localclaude). -
specdrift render --digest FILE --out docs/rules/upstream-spec.md:title: Upstream specfrontmatter; one section per artifact kind listing documented fields and enums, whether claudelint parses each, and the acknowledgement reason where present; a “verified against Claude Code vX.Y.Z” line frommeta; tables emitted in one consistent pipe style sojust lint-md(MD060) passes; the page lands in the StarlightRulessidebar group automatically and links to the deprecated-and-removed tools table in rules.md rather than duplicating it. -
render --check(exit 1 when the committed page differs from a fresh render) wired per OQ9; commit the first rendered page. -
claudelint versionthird line per OQ7 andrules --jsonupstream_version(additive);docs/rules-json-schema.mdupdated;internal/clireadsupstream.LoadEmbedded()(import directioncli → upstream;upstreamimports nothing fromcli,engine, orrules). - CLAUDE.md “Project status” paragraph and the
versionoutput shape note ininternal/cli/version.goupdated; DESIGN-0006 gains an “Implemented by IMPL-0005” note and moves to Implemented. - Dogfood:
just self-check; confirmclaudelint versionon the release binary prints the spec line. - Flip this doc to Completed; PR labelled
minor;just cigreen.
Success Criteria
Section titled “Success Criteria”- On merge day the runtime job reports zero disagreements with the
latest published CLI, and the
skills/probe confirmscontentsis empty (or the report says the assumption changed). docs/rules/upstream-spec.mdrenders in both Starlight and MkDocs, passesjust lint-md, andrender --checkfails when the digest changes without a re-render.claudelint versionprints the spec line;rules --jsoncarriesupstream_versionand still matchesdocs/rules-json-schema.md.just cigreen; minor release ships; DESIGN-0006 Implemented; IMPL-0005 Completed.
Phase 3 results (2026-09-15)
Section titled “Phase 3 results (2026-09-15)”validate-fixtures agrees on all six fixtures against two runtimes: the
locally installed CLI at v2.1.259, and @latest at v2.1.272 in the PR
run of spec-drift.yml (run 34997855440). The skills/ probe confirms
contents is still empty in both, so the coverage claim holds.
That PR run also proves the Phase 3 workflow wiring end to end without
waiting for merge: mise installed the pinned Node, npm installed the
CLI, validate-fixtures produced its report, and the append and decide
steps folded it into the drift report.
Probing the real validator turned up three things the published tables
do not say, all on command marketplace sources:
| Finding | Detail |
|---|---|
timeout unit and bound | Seconds, capped at 600. The docs give neither. |
timeout type | Must be a number; the quoted form the parser tolerates is rejected. |
mode | Documented as optional, rejected by v2.1.259. |
marketplace/plugin-source-valid deliberately does not enforce the
600-second cap. A limit claudelint cannot cite to the documentation is a
limit the guardrail cannot keep honest, and the rule would then be
asserting a fact no committed artifact records. The mode rejection is
a docs-versus-runtime disagreement rather than a claudelint bug, and it
is the exact class of finding the runtime job exists to surface.
The rendered page builds in both pipelines: Starlight (just docs-build,
37 pages) and MkDocs (just docs-mkdocs-check, strict). render --check
was verified in both directions by removing one tool from the digest and
restoring it.
render --check now guards the page three ways: the ci.yml lint job,
just docs-check, and a Go test comparing the committed page with a
fresh render. That last one is the only guard that runs without anyone
remembering to.
File Changes
Section titled “File Changes”| File | Action | Description |
|---|---|---|
internal/upstream/source.go | Create | Source table (ids, tiers, URLs) and schema probes |
internal/upstream/fetch.go | Create | Context-aware fetcher with retries, timeout, manifest |
internal/upstream/table.go | Create | GFM table scanner and section scoping |
internal/upstream/digest.go, lock.go, diff.go, disagreements.go | Create | Digest types and emitter, lock, structural diff and renderers |
internal/upstream/extract_*.go | Create | One extractor per source (skills, agents, plugins, marketplaces, hooks, mcp, tools, schemastore, portable, changelog) |
internal/upstream/command.go | Create | cobra wiring for pull, digest, diff, check, render, validate-fixtures |
internal/upstream/render.go, validate.go | Create | Phase 3: docs page renderer, runtime-validator driver |
internal/upstream/guard_test.go | Create | Phase 2 guardrail |
internal/upstream/spec/spec.go, spec/digest.json | Create | Leaf package holding the embedded digest and its version line, so the release binary does not import the fetcher (Phase 3) |
internal/upstream/sources.lock.json | Create | Committed source lock |
internal/upstream/coverage.go | Create | The one file importing internal/artifact; what the renderer means by “claudelint reads this” (Phase 3) |
internal/upstream/acknowledged.json, runtime_fixtures.json | Create | Deliberate deviations; runtime fixture manifest |
internal/upstream/testdata/snippets/** | Create | Golden section snippets per extractor |
cmd/specdrift/main.go | Create | Thin main |
internal/artifact/parse_md_kinds.go, parse_json.go, parse_marketplace.go, types.go, knowndata.go | Modify | Exported key lists; archive and command source kinds; hook events and tools refresh; DeprecatedTools |
internal/rules/agents/toolsknown.go, internal/rules/commands/allowedtoolsknown.go | Modify | Removed/renamed wording from DeprecatedTools |
internal/rules/marketplace/reservedname.go, pluginsourcevalid.go, externalsourceskipped.go | Modify | claude-tag-plugins; archive and command handling |
internal/rules/version.go | Modify | Ruleset bump; doc comment covers known-data changes |
internal/cli/version.go, rules.go | Modify | Spec line; upstream_version |
internal/rules/mcp/transportknown.go | Modify | KnownTransports moved to internal/artifact with the rest of the canonical data |
internal/artifact/testdata/ok/**, bad/** | Create | New-event hooks file, new-tool agent and skill, archive-and-command marketplace, runtime fixtures |
.github/workflows/spec-drift.yml | Create | Weekly drift workflow |
scripts/spec-drift-issue.sh, scripts/labels.sh | Create / Modify | Issue lifecycle; spec-drift label |
justfile | Modify | spec-check, spec-sync, spec-validate-fixtures; render --check in docs-check |
.github/workflows/ci.yml | Modify | render --check step (Phase 3, per OQ9) |
docs/rules/upstream-spec.md | Create | Rendered digest page |
docs/rules/rules.md, README.md, docs/rules-json-schema.md, CLAUDE.md | Modify | Event and tool tables, reserved names, source kinds, JSON field, commands and conventions |
docs/design/0006-*.md | Modify | §7 rows for HookEntryKeys and DeprecatedTools; implemented-by note; status Implemented |
mkdocs.yml | Modify | Nav entry for the rendered spec page (the Starlight sidebar picks it up automatically) |
Testing Plan
Section titled “Testing Plan”- Unit (library): table scanner edge cases; every extractor against its golden snippet plus the three negative cases; digest determinism; diff goldens; lock diff classification.
- Fetch:
httptest.Serverfor retry, timeout, redirect, and manifest fields; a test asserting no extractor or command test opens a real socket (GOFLAGS=-mod=modis irrelevant; simply no network hosts in test config). - Command:
checkend to end againsthttptest-served fixtures for exit 0, 1, and 2;--updatewrites both files;render --checkfor both outcomes;validate-fixtureswith a fakeclaudeonPATH. - Guardrail: one subtest per row; message contents asserted; stale acknowledgement fails; unknown acknowledgement path fails at load.
- Parsers and rules (Phase 2): table-driven tests for archive and command sources with ranges; reserved-name addition; regression fixtures for unknown event, unknown tool, empty source still flag.
- Workflow:
just lint-actions; the three-run dispatch sequence on the branch;scripts/spec-drift-issue.sh --dry-runoutput checked by eye for each branch of the state machine. - Coverage:
just coverage-gate(55% floor) oninternal/upstreamand every touchedinternal/package. - Docs:
just lint-md,just docs-check,just docs-mkdocs-checkafter Phase 3’s rendered page lands. - Dogfood:
just self-checkplus adonaldgifford/claude-skillscheckout after Phases 2 and 3.
Dependencies
Section titled “Dependencies”- DESIGN-0006 approved (all twelve questions resolved 2026-09-12).
- GitHub Actions network egress to
code.claude.com,www.schemastore.org,raw.githubusercontent.com, and the npm registry (Phase 3);ghon the runner (preinstalled onubuntu-latest). - Node pinned in
mise.toml(already, for the docs site) for the runtime validator install. - No new Go module dependencies: cobra is already present; the table scanner is hand-written.
- Upstream shape as of 2026-09-13: 33 hook events, 45 tools, six
marketplace source kinds, 17 reserved names, docs markers through
v2.1.268. Re-run
spec-checkat the start of each phase; the counts in this document are the day-one expectations, not fixed targets.
Resolved Decisions
Section titled “Resolved Decisions”All eleven resolved by Donald on 2026-09-13, every one on option (a). Task references such as “per OQ2” point here.
- OQ1 — Phase 1 release label:
dont-release. goreleaser builds only./cmd/claudelint, and Phase 1 does not touch it. - OQ2 — the four undocumented entries in
KnownTools: keepTaskwith an acknowledgement citing the v2.1.63 rename toAgent; dropBashOutput,KillShell, andMultiEdit, since the tools-known rules exist precisely because the runtime silently ignores unknown names. Addendum: record deprecated, removed, and renamed tools with the version and replacement in a “Deprecated and removed tools” table in rules.md, backed byartifact.DeprecatedToolsso the rules can name the replacement and the guardrail can catch a stale row. The Claude Code changelog recordsBashOutput(unshipped 2.0.64, replaced byTaskOutput) andTaskOutput(deprecated 2.1.83 in favour ofRead) but has no entry forKillShell,MultiEdit, or theTaskrename, so the table carries a source column and falls back to the first docs marker at which the tool left the reference table. - OQ3 — ruleset version for Phase 2: minor,
v1.5.0→v1.6.0; theRulesetVersiondoc comment gains known-data changes as an explicit bump trigger. - OQ4 — documented fields no rule consumes: acknowledge each with the reason “no consuming rule; parse when a rule needs it”. Parsing follows rule demand in the IMPL that adds the rule.
- OQ5 — runtime fixture layout: reuse the two existing
.claude-pluginfixtures and add the missingpass/failcases as.claude-plugin-shaped directories underinternal/artifact/testdata, shared with the parser tests. - OQ6 — golden snippets:
specdrift digest --write-snippets DIRwrites exactly the section bytes each extractor consumed. - OQ7 — version spec line:
spec v2.1.268 (a1b2c3d4), the parenthesised value being the first eight hex characters of the digest’s sha256. No date. - OQ8 — sanity floors: derived at run time from the committed
digest as
max(1, ceil(0.75 × committed count)). - OQ9 — where
render --checkruns: a step inci.yml’slintjob plus the same command insidejust docs-check. - OQ10 — runtime validator wiring: one job; the runtime steps run
after the drift check with
continue-on-error, their report is appended, and the issue script runs once. - OQ11 — marketplace
archiveandcommandsource kinds: fix in Phase 2 as part of the catch-up.
Amendments made while implementing
Section titled “Amendments made while implementing”- Lock fields (Phase 1). DESIGN §4 specified
last_modifiedper source and made no exception for the optional probes. Measuring both against the live sources showed each one breaks the invariant the lock exists for.code.claude.comsetsLast-Modifiedto the time of the request, so two fetches five seconds apart differ for byte-identical pages; the twocode.claude.com/schemasprobes resolve to a product page whose body carries a per-request nonce. Both were dropped from the committed lock and DESIGN §4 was amended in the same commit. The header is still recorded in the work-directorymanifest.json.
Open Questions
Section titled “Open Questions”None. All eleven were resolved on 2026-09-13; see Resolved Decisions.
References
Section titled “References”- DESIGN-0006 — the design this implements; §1 sources, §3 extractors, §6 workflow, §7 guardrail, Resolved Decisions
- INV-0006 — the manual audit whose approach the tool automates
- IMPL-0004 — house checklist, dogfood convention, and OQ6 release cadence reused here
- DESIGN-0005 — agent enum sets under guard
internal/rules/all/fingerprint_test.goandcmd/genfp/main.go— guardrail and dev-tool precedents- Claude Code docs (raw Markdown): skills, sub-agents, plugins reference, plugin marketplaces, hooks, MCP, tools reference
- SchemaStore: plugin manifest, marketplace, settings
- Agent Skills specification and skills-ref validator; Anthropic quick_validate.py