Website checker: reads robots.txt, then a site's home page and a few linked pages, on user request or a monthly refresh.
- Category
- Operator
- CheckAnySite
- AccessIndicates who drives the bot's activity. Bots with "direct" access are controlled directly by their operator, while bots with "intermediary" access can be controlled by end users.
- Intermediary
- Documentation
- https://checkanysite.com/bot
- Signature Agent URLLearn more...
- https://checkanysite.com/.well-known/http-message-signatures-directory