Tekodot Ltd's search crawler. Obeys robots.txt and crawl delays; stops on blocks; authenticates with Web Bot Auth.
- Category
- Operator
- Tekodot Ltd
- AccessIndicates who drives the bot's activity. Bots with "direct" access are controlled directly by their operator, while bots with "intermediary" access can be controlled by end users.
- Direct
- Documentation
- https://crawl.prebrief.co/
- Signature Agent URLLearn more...
- https://crawl.prebrief.co/.well-known/http-message-signatures-directory