Scorecard schema

A scorecard is the structured output of anc check <binary> --output json. The site reads scorecards under scorecards/ and renders them at /scorecards and at each tool's /score/<name> page. This page documents every field that appears in a scorecard, what it means, and where it comes from. It is intentionally exhaustive: anything in the JSON should be explainable from this page.

Filename

Scorecards are stored on disk as:

scorecards/<name>-v<version>.json

Where <name> matches the registry's name field (URL slug) and <version> is the SemVer string captured at scoring time. The filename's <version> segment is the canonical version anchor — the site reads it directly off disk and displays it as the scored version on every per-tool page. The scorecard's tool.version field (added in schema 0.4) is informational; when both are present and disagree, the build aborts with an integrity error. The filename never lies.

Top-level fields

{
  "schema_version": "0.4",
  "spec_version": "...",
  "tool":   { "name": "...", "binary": "...", "version": "..." },
  "anc":    { "version": "..." },
  "run":    { "invocation": "...", "started_at": "...", "duration_ms": 0,
              "platform": { "os": "...", "arch": "..." } },
  "target": { "kind": "...", "path": null, "command": "..." },
  "audience": "...",
  "audience_reason": null,
  "audit_profile": null,
  "summary": { ... },
  "coverage_summary": { ... },
  "results": [ ... ]
}

tool

Self-describing tool identity. Lets a downstream consumer answer "what was scored?" without cross-referencing the registry.

"tool": {
  "name": "rg",
  "binary": "rg",
  "version": "ripgrep 15.1.0"
}

Build-time invariant: when tool.version contains a SemVer-shaped token (X.Y or X.Y.Z), it must equal the filename version. Drift fails the build with a parser-asymmetry error — the regen script's version_extract snippet and the CLI's internal probe are the only two places that derive a version from the binary, and they must agree.

anc

Provenance for the scorecard: which anc build produced it.

"anc": {
  "version": "0.2.1"
}

The per-tool page renders anc.version.

run

Run-context: the literal invocation, when it ran, how long it took, and what platform it ran on.

"run": {
  "invocation": "anc check --command rg --output json",
  "started_at": "2026-04-30T04:18:53.099683344Z",
  "duration_ms": 53,
  "platform": { "os": "linux", "arch": "x86_64" }
}

Security note — run.invocation: for command-mode runs the invocation is the canonical anc check --command <name> [--audit-profile <X>] [--output json] shape, which is safe to embed publicly. For project-mode runs (target.kind: "project") the invocation may include a local filesystem path (anc check ./local/repo); the site falls back to the synthesized form for those runs to avoid leaking machine-local paths into HTML, markdown, and /llms-full.txt. Mirror this gate downstream if you fetch the JSON directly.

target

What the run was scoring: a command, a binary on disk, or a project tree.

"target": {
  "kind": "command",
  "path": null,
  "command": "rg"
}

Security note — target.path: when kind is project, this can carry a local directory path (/home/me/dev/foo). It is not currently rendered on any per-tool page (every leaderboard entry today is command-mode), but downstream consumers reading the JSON should treat it as machine-local.

summary

Counts of the checks the runner actually executed. Adds up to total.

"summary": {
  "total": 11,
  "pass": 8,
  "warn": 0,
  "fail": 0,
  "skip": 3,
  "error": 0
}

The headline score on the leaderboard is pass / (pass + warn + fail) — skip and error are excluded from the denominator on purpose, as documented on the methodology page.

coverage_summary

Tally of how much of the agentnative spec the run verified. Distinct from summary, which counts the runner's checks. One implemented check can verify zero, one, or many spec requirements; many spec requirements aren't yet covered by any implemented check.

"coverage_summary": {
  "must":   { "total": 23, "verified": 9 },
  "should": { "total": 16, "verified": 0 },
  "may":    { "total": 7,  "verified": 0 }
}

If coverage_summary.must.verified is below summary.pass, that's expected — a single passing check can map to multiple MUSTs. If should.verified and may.verified are zero across the board, that's also expected: those tiers are aspirational and will fill in as the runner grows checks mapped to them.

results

Array of one object per check the runner attempted. Order is stable across runs of the same anc version.

{
  "id": "p3-help",
  "label": "Help flag produces useful output",
  "group": "P3",
  "layer": "behavioral",
  "status": "pass",
  "evidence": null,
  "confidence": "high"
}

status semantics in detail

• pass — Check ran, found no issue.
• warn — Check ran, found a soft signal worth noting. Lowers the headline score.
• fail — Check ran, found a hard non-compliance. Lowers the headline score.
• skip — Check did not run, by design. Either the tool's surface made it inapplicable (e.g., a --help parser check on a tool with no flags) or the active audit_profile suppressed it (evidence will name the profile).
• error — Check tried to run and crashed before producing a verdict. Treated as no-signal, not a defect.

What is not in the scorecard (yet)

The site is transparent about gaps that future schema bumps may fill. Schema 0.4 closed the tool-identity / generated-at gap (see tool, anc, run, target above). Still outstanding today:

• Per-check timing — run.duration_ms is the wall-clock total for the run, not per-check. Individual check timings are observable from the runner's stdout but not captured in the JSON.
• Editorial fields inside the scorecard — tier, language, creator, description, install, repo/url remain in the registry. Migrating them into the scorecard would let the registry shrink to a name list; deferred to a future schema bump.

When the anc CLI grows fields for any of the above, the schema page above will be updated and schema_version will bump.

Why is audience null for some tools?

When a tool is scored with audit_profile: human-tui, anc suppresses one or more of the four signal checks the audience classifier consumes. With incomplete inputs, the classifier emits audience: null and sets audience_reason: "suppressed" rather than guess. This is correct, intentional, and shared across every TUI on the leaderboard. The methodology page covers the full classifier definition.