llm

Access large language models from the command-line

notable Python simonw/llm
68% pass rate
3/8 principles met

Spec Coverage

How many of the spec's requirements were verified for this tool. See /coverage for the full matrix.

LevelTotalVerifiedUnverified
MUST28199
SHOULD21138
MAY10100

Audience signal: mixed

This tool sends mixed signals: some agent-readable affordances are present, others are not. Treat the warnings below as friction points, not defects.

This is an informational signal, not an authoritative verdict — see methodology. The per-audit evidence below is the ground truth.

Top Issues

All Audits

P1: Non-Interactive by Default

PASSNon-interactive by default
SKIPNon-interactive gate flag advertised in --helptarget satisfies P1 via alternative gate (help-on-bare or stdin-primary)
WARNFlags advertise env-var bindings in --help2 flag(s) found in --help but no `[env: NAME]` bindings advertised
PASSSecret-bearing flags expose stdin or *-file companion
WARN`--help` advertises default values for flagsno default-value annotations found in --help. SHOULD-tier — agents reading help text need to see what value a flag falls back to when omitted (`[default: <value>]` per clap convention).
WARNRich-TUI affordance for TTY contextsno rich-TUI affordance detected (no `--tui`/`--interactive`/`--ui` flag, no spinner/progress/tui mention in --help). MAY-tier — rich TUI in TTY contexts is a nice-to-have, not required.

P2: Structured, Parseable Output

WARNStructured output support--output/--format flag detected but could not validate JSON via safe probes (--help/--version override output flags in most CLIs)
SKIPStructured-output CLI exposes its schema at runtimeno structured-output indicator (--output / --format / json / jsonl) in --help
WARN--json / --jsonl short aliases for --outputno --json or --jsonl short alias found. Agents and pipelines benefit from short forms alongside the canonical `--output` enum.
WARN`--raw` flag for pipe-safe unformatted outputno `--raw` flag advertised. MAY-tier — useful for pipelines that want to strip formatting before piping to other tools.
SKIP`--output` advertises additional formats beyond text/jsonno `--output` or `--format` flag advertised; vacuous skip for MAY-tier extra formats.
PASSBad invocation exits with structured usage-error code (2)
SKIPErrors emit JSON envelope with `error`/`kind`/`message` under `--output json`binary does not advertise `--output json` in --help; MUST applies only to CLIs that opt into the JSON contract.
SKIPJSON success and error envelopes share their non-payload key setbinary does not advertise `--output json` in --help; envelope-consistency only applies to CLIs that opt into the JSON contract.

P3: Progressive Help Discovery

PASSHelp flag produces useful output
WARNVersion flag works (`--version` plus short alias)`--version` works but no short alias responded (tried -V, -v, -version). Adding one shortens version probes for agents.
WARNVersion flag works (`--version` plus short alias)`--version` works but no short alias responded (tried -V, -v, -version). Adding one shortens version probes for agents.
WARN`examples` subcommand or `--examples` flag for curated usage patternsno `examples` subcommand or `--examples` flag found. MAY-tier — a curated usage block keeps agents from hunting through long help text.
WARNShort `-h` summary differs from `--help` long form`-h` and `--help` produce byte-identical output. SHOULD-tier — clap renders the short summary on `-h` and the full description on `--help` when `long_about` is set; collapsing them gives agents no concise list-level grep target.
FAILEach subcommand's `--help` ships at least one invocation examplesubcommands missing example invocations in their `--help`: aliases, chat, collections, embed, embed-models, fragments, install, keys, logs, models, openai, plugins, schemas, templates, tools, uninstall. Examples teach agents the call shape faster than option tables; use clap's `after_help` or a dedicated `Examples:` block.
WARNHelp text pairs human and `--output json` example invocationsno paired text + `--output json` example found within 5 lines in top-level or any subcommand `--help`. Pairing keeps agents from reverse-engineering the JSON invocation from the text one.

P4: Fail-Fast, Actionable Errors

PASSRejects invalid arguments
PASSError messages include a hint or remediation phrase
SKIP`--output json` produces JSON-formatted errorsbinary does not advertise `--output json` in --help; SHOULD applies only to CLIs that opt into the JSON contract.

P5: Safe Retries & Mutation Boundaries

SKIPDestructive subcommands require `--force` or `--yes`no destructive subcommands detected; MUST applies conditionally to CLIs with destructive operations.
SKIPRead and write surfaces are both visible in subcommand listno recognizable read or write subcommand verbs; the read/write distinction is unobservable from the help surface alone.

P6: Composable, Predictable Command Structure

PASSHandles SIGPIPE gracefully
SKIPPager-using CLI ships --no-pager escape hatchno pager signal (less/more/$PAGER/--pager) in --help
PASSRespects NO_COLOR
WARNSubcommand verbs follow community-standard names3/18 subcommand(s) follow standard verb names. Non-standard: aliases, chat, collections, embed, embed-models, embed-multi, fragments, keys, models, openai, plugins, schemas, similar, templates, tools. MAY-tier — community-standard verbs (get/list/create/update/delete) help agents predict subcommand behavior across CLIs.
WARN`--color` flag for explicit color controlno `--color` flag advertised. MAY-tier — `auto|always|never` lets agents and pipelines override the TTY-based default.
SKIPInput-accepting commands read from stdin when no file is givenno input-accepting subcommand detected (process/parse/convert/transform/analyze/validate/format/lint/audit); vacuous skip for the conditional SHOULD.
WARNSubcommand naming follows a consistent verb/noun conventionsubcommand naming is inconsistent: 7 non-verb subcommand(s) (aliases, collections, fragments, keys, logs, schemas, templates) mix verb and non-verb children at the second level, so an agent cannot predict where the action lives. SHOULD-tier: pick a consistent shape (all verb-first, all noun-verb hierarchy, or any combination where each non-verb group's children are uniformly verbs). The verb list is a heuristic; inspect `--help` to confirm.
PASSOperations are subcommands, not verb-shaped flags

P7: Bounded, High-Signal Responses

WARNQuiet mode availableno --quiet/-q flag detected in --help output
WARN`--verbose` flag for diagnostic escalationno `--verbose` / `-v` flag advertised. SHOULD-tier — agents debugging failures need a way to escalate diagnostic detail.
SKIP`--limit` / `--max-results` flag for list operationsno list-style subcommand detected (list/ls/search/query/find/show/get); vacuous skip for the list-only SHOULD.
SKIPCursor-based pagination flags for list traversalno list-style subcommand detected; vacuous skip for the list-only MAY.
SKIP`--timeout` flag for long-running operationsno long-running subcommand detected (serve/daemon/watch/tail/monitor/follow/run/start/stream); vacuous skip for the conditional SHOULD.
WARNHelp text advertises TTY-aware verbosity behaviorno TTY-aware language found in `--help`. MAY-tier — automatic verbosity reduction when stdout is piped or redirected lets agents skip the explicit `--quiet` flag. Behavioral probes cannot simulate a real TTY without a pty crate, so this audit relies on documented intent.

P8: Discoverable Through Agent Skill Bundles

PASSSkill bundle has install path (`tool skill install [<host>]`)
PASS`skill install --all` for multi-runtime install
PASS`skill update` / `skill upgrade` for bundle refresh

Details

Version scored
0.31
Audit date
2026-06-01 17:36:00 UTC
Duration
16.4s
Platform
linux/x86_64
Mode
command
Anc build
0.5.0
Install
brew install llm

Embed the badge

The badge floor is 70%; this scorecard is at 68% (2 points below). Once the score clears the floor, the embed snippet will appear here. The top issues above are the place to start.

Reproduce this scorecard for llm locally and inspect the failing audits:

anc audit --command llm --output json

Install anc first if you don't have it. Add --output json to get the same JSON shape committed under scorecards/.