Robots.txt Tester

Check if your URL is crawlable by search engines

Enter any URL to check whether Googlebot, Bingbot, and other crawlers can access it. We'll fetch the robots.txt file and evaluate all applicable rules.