Crawler available to site owners to request crawls of their own sites for targeted AI training
- Category
- AI Crawler
- Operator
Crawler available to site owners to request crawls of their own sites for targeted AI training
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/[CHROME_VERSION] Mobile Safari/537.36 (compatible; Google-CloudVertexBot; +https://cloud.google.com/enterprise-search)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Google-CloudVertexBot; +https://cloud.google.com/enterprise-search) Chrome/[CHROME_VERSION] Safari/537.36
User-Agent: CloudVertexBot Disallow: /