ClaudeBot

If you block it

Your content is not collected for training. Claude can still reach your page when a person asks about it — that is Claude-User, and Anthropic says that one honors robots.txt too.

What Anthropic says it is for

Collecting web content that could potentially contribute to training Anthropic’s models.

Published details

robots.txt

Says it respects robots.txt“Anthropic says its crawlers respect “do not crawl” signals by honoring industry standard directives in robots.txt — a statement that covers all of its bots.”

User agent

Not published as a full string. Anthropic publishes only the token.Match on the prefix ClaudeBot rather than the whole string: operators version these, and an exact match stops working on the next release.

Verification

• Published address list: https://claude.com/crawling/bots.json

Anthropic does not publish full user agent strings or a reverse-DNS hostname. The authoritative identifier is the IP list, shared by all three of its crawlers.

Not addressed in the documentation

• As of 2026年10月12日, Anthropic’s documentation does not address a full user agent string.
• As of 2026年10月12日, Anthropic’s documentation does not address a reverse-DNS hostname to check against.