AboutCopy linkClaude-SearchBot navigates the web to improve search result quality for Claude users.CategoryAI SearchOperatorAnthropicDocumentationhttps://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler
User agentCopy linkHTTP requestsClaude-SearchBotrobots.txtLearn more...CopyUser-Agent: Claude-SearchBot Disallow: /
HTTP traffic from Claude-SearchBotCopy linkHTTP requests from the specified bot over the selected time periodLearn more...Share this...More actions...