Korean search engine crawler
- HTTP 요청
Mozilla/5.0 (Unknown; Linux x86_64) AppleWebKit/538.1 (KHTML, like Gecko) Safari/538.1 Daum/4.1
- robots.txt자세히 알아보기
User-Agent: Daum/4.1 Disallow: /
Korean search engine crawler
Mozilla/5.0 (Unknown; Linux x86_64) AppleWebKit/538.1 (KHTML, like Gecko) Safari/538.1 Daum/4.1
User-Agent: Daum/4.1 Disallow: /