# Fix: /robots.txt present

> Web-audit fix skill for the `robots` check (Discoverability, SHOULD).

## Goal

Publish robots.txt and state your crawl policy explicitly.

## Fix

Publish a `robots.txt`. Declare your crawl and AI-usage policy explicitly rather than leaving
it to inference. An absent `robots.txt` is ambiguous; a present one is a clear signal.

## Resources

- [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309)

## Copy-paste prompt

Paste this into your coding agent, replacing the Issue line with the Result line from [your audit](https://anc.dev/web-audit):

```text
Goal: Publish robots.txt and state your crawl policy explicitly
Issue: <the audit's finding for this check>
Fix: Publish a `robots.txt`. Declare your crawl and AI-usage policy explicitly rather than leaving it to inference. An absent `robots.txt` is ambiguous; a present one is a clear signal.
Skill: https://anc.dev/web-audit/skill/robots
Docs: https://www.rfc-editor.org/rfc/rfc9309
```

## Verify

Re-run the audit at [https://anc.dev/web-audit](https://anc.dev/web-audit) or call the `audit_website` MCP tool; the `robots` check should report `pass`.