The Cloudflare Crawler is a well-behaved crawler that retrieves web content. By default, it self-identifies as a bot, honors robots.txt directives, and cannot bypass CAPTCHAs or bot protection. Used by Cloudflare customers via the Browser Run /crawl endpoint.
- 類別
- AI 網路爬蟲
- 運算子
- Cloudflare
- 簽名智能體 URL代理程式在 Web Bot Auth 下發布的 HTTP 訊息簽章金鑰目錄 URL。原始伺服器會取得此目錄以驗證代理程式 HTTP 請求上的簽章。
- https://web-bot-auth-crawl.cloudflare-browser-rendering-085.workers.dev/.well-known/http-message-signatures-directory