Robots.txt Checker
Fetch, read, and understand a site's robots.txt in seconds.
We automatically fetch /robots.txt. Works with domains, homepages, or any URL.