Glossary
/// Definition
ClaudeBot
ClaudeBot is Anthropic's crawler for content used in Claude's answers and training.
In depth
ClaudeBot (and Claude-Web) crawls public web content to power Anthropic's Claude assistant. Sites can control access via robots.txt.