Brave search has a crawler to discover new pages and index their content.
- Category
- AI Search
- Operator
- Brave Software, Inc.
- HTTP requests
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Bravebot/1.0; +https://search.brave.com/help/brave-search-crawler) Chrome/W.X.Y.Z Safari/537.36
- robots.txtLearn more...
User-Agent: Bravebot Disallow: /