Spec Coverage Matrix
Which requirements have automated checks — and which remain uncovered.
Generated by anc generate coverage-matrix from the
agentnative principle registry.
Summary
46
total requirements
19
covered
27
uncovered
| Level |
Total |
Covered |
Uncovered |
| MUST |
23 |
17 |
6 |
| SHOULD |
16 |
2 |
14 |
| MAY |
7 |
0 |
7 |
| Level |
Requirement |
Applicability |
Verified by |
| MUST |
Every flag settable via environment variable (falsey-value parser for booleans). |
Universal |
p1-env-hints p1-env-flags-source |
| MUST |
`--no-interactive` flag gates every prompt library call; when set or stdin is not a TTY, use defaults/stdin or exit with an actionable error. |
Universal |
p1-non-interactive p1-flag-existence p1-non-interactive-source |
| MUST |
Headless authentication path (`--no-browser` / OAuth Device Authorization Grant). |
CLI authenticates against a remote service |
p1-headless-auth |
| SHOULD |
Auto-detect non-interactive context via TTY detection; suppress prompts when stderr is not a terminal. |
Universal |
p1-tty-detection-source |
| SHOULD |
Document default values for prompted inputs in `--help` output. |
Universal |
UNCOVERED |
| MAY |
Rich interactive experiences (spinners, progress bars, menus) when TTY is detected and `--no-interactive` is not set. |
Universal |
UNCOVERED |
| Level |
Requirement |
Applicability |
Verified by |
| MUST |
`--output text|json|jsonl` flag selects output format; `OutputFormat` enum threaded through output paths. |
Universal |
p2-json-output p2-structured-output |
| MUST |
Data goes to stdout; diagnostics/progress/warnings go to stderr — never interleaved. |
Universal |
p2-output-module |
| MUST |
Exit codes are structured and documented (0 success, 1 general, 2 usage, 77 auth, 78 config). |
Universal |
UNCOVERED |
| MUST |
When `--output json` is active, errors are emitted as JSON (to stderr) with at least `error`, `kind`, and `message` fields. |
Universal |
UNCOVERED |
| SHOULD |
JSON output uses a consistent envelope — a top-level object with predictable keys — across every command. |
Universal |
UNCOVERED |
| MAY |
Additional output formats (CSV, TSV, YAML) beyond the core three. |
Universal |
UNCOVERED |
| MAY |
`--raw` flag for unformatted output suitable for piping to other tools. |
Universal |
UNCOVERED |
| Level |
Requirement |
Applicability |
Verified by |
| MUST |
Every subcommand ships at least one concrete invocation example (`after_help` in clap). |
CLI uses subcommands |
UNCOVERED |
| MUST |
The top-level command ships 2–3 examples covering the primary use cases. |
Universal |
p3-help |
| SHOULD |
Examples show human and agent invocations side by side (text then `--output json` equivalent). |
Universal |
UNCOVERED |
| SHOULD |
Short `about` for command-list summaries; `long_about` reserved for detailed descriptions visible with `--help`. |
Universal |
UNCOVERED |
| MAY |
Dedicated `examples` subcommand or `--examples` flag for curated usage patterns. |
Universal |
UNCOVERED |
| Level |
Requirement |
Applicability |
Verified by |
| MUST |
Parse arguments with `try_parse()` instead of `parse()` so `--output json` can emit JSON parse errors. |
Universal |
p4-try-parse |
| MUST |
Error types map to distinct exit codes (0, 1, 2, 77, 78). |
Universal |
p4-bad-args p4-exit-codes |
| MUST |
Every error message contains what failed, why, and what to do next. |
Universal |
UNCOVERED |
| SHOULD |
Error types use a structured enum (via `thiserror` in Rust) with variant-to-kind mapping for JSON serialization. |
Universal |
p4-error-module p4-error-types |
| SHOULD |
Config and auth validation happen before any network call (three-tier dependency gating). |
CLI makes network calls |
UNCOVERED |
| SHOULD |
Error output respects `--output json`: JSON-formatted errors go to stderr when JSON output is selected. |
Universal |
UNCOVERED |
| Level |
Requirement |
Applicability |
Verified by |
| MUST |
Destructive operations (delete, overwrite, bulk modify) require an explicit `--force` or `--yes` flag. |
CLI has destructive operations |
UNCOVERED |
| MUST |
The distinction between read and write commands is clear from the command name and help text alone. |
CLI has both read and write operations |
UNCOVERED |
| MUST |
A `--dry-run` flag is present on every write command; dry-run output respects `--output json`. |
CLI has write operations |
p5-dry-run |
| SHOULD |
Write operations are idempotent where the domain allows it — running the same command twice produces the same result. |
CLI has write operations |
UNCOVERED |
| Level |
Requirement |
Applicability |
Verified by |
| MUST |
SIGPIPE fix is the first executable statement in `main()` — piping output to `head`/`tail` must not panic. |
Universal |
p6-sigpipe |
| MUST |
TTY detection plus support for `NO_COLOR` and `TERM=dumb` — color codes suppressed when stdout/stderr is not a terminal. |
Universal |
p6-no-color-behavioral p6-no-color p6-no-color |
| MUST |
Shell completions available via a `completions` subcommand (Tier 1 meta-command — needs no config/auth/network). |
Universal |
p6-completions |
| MUST |
Network CLIs ship a `--timeout` flag with a sensible default (e.g., 30 seconds). |
CLI makes network calls |
p6-timeout |
| MUST |
If the CLI uses a pager (`less`, `more`, `$PAGER`), it supports `--no-pager` or respects `PAGER=""`. |
CLI invokes a pager for output |
p6-no-pager-behavioral p6-no-pager |
| MUST |
Agentic flags (`--output`, `--quiet`, `--no-interactive`, `--timeout`) are `global = true` so they propagate to every subcommand. |
CLI uses subcommands |
p6-global-flags |
| SHOULD |
Commands that accept input read from stdin when no file argument is provided. |
CLI has commands that accept input data |
UNCOVERED |
| SHOULD |
Subcommand naming follows a consistent `noun verb` or `verb noun` convention throughout the tool. |
CLI uses subcommands |
UNCOVERED |
| SHOULD |
Three-tier dependency gating: Tier 1 (meta) needs nothing, Tier 2 (local) needs config, Tier 3 (network) needs config + auth. |
Universal |
UNCOVERED |
| SHOULD |
Operations are modeled as subcommands, not flags (`tool search "q"`, not `tool --search "q"`). |
CLI performs multiple distinct operations |
UNCOVERED |
| MAY |
`--color auto|always|never` flag for explicit color control beyond TTY auto-detection. |
Universal |
UNCOVERED |
| Level |
Requirement |
Applicability |
Verified by |
| MUST |
A `--quiet` flag suppresses non-essential output; only requested data and errors appear. |
Universal |
p7-quiet |
| MUST |
List operations clamp to a sensible default maximum; when truncated, indicate it (`"truncated": true` in JSON, stderr note in text). |
CLI has list-style commands |
p7-output-clamping |
| SHOULD |
A `--verbose` flag (or `-v` / `-vv`) escalates diagnostic detail when agents need to debug failures. |
Universal |
UNCOVERED |
| SHOULD |
A `--limit` or `--max-results` flag lets callers request exactly the number of items they want. |
CLI has list-style commands |
UNCOVERED |
| SHOULD |
A `--timeout` flag bounds execution time so agents are not blocked indefinitely. |
Universal |
UNCOVERED |
| MAY |
Cursor-based pagination flags (`--after`, `--before`) for efficient traversal of large result sets. |
CLI returns paginated results |
UNCOVERED |
| MAY |
Automatic verbosity reduction in non-TTY contexts (same behavior `--quiet` explicitly requests). |
Universal |
UNCOVERED |
About this matrix
This matrix is generated from the agentnative principle
registry — a flat list of every MUST, SHOULD, and MAY across the seven principles, each mapped to the check IDs
that verify it. Requirements with no verifiers are marked UNCOVERED: the spec requires them but
anc does not yet test for them.
"Covered" means a check exists that targets this requirement. "Verified" (on scorecard pages)
means the check actually ran for a specific tool. A requirement can be covered in the matrix but unverified for a
particular tool if the check was skipped (e.g., source checks on a binary-only audit).