How to use Robots.txt Tester
- Paste a robots.txt document.
- Enter a crawler user-agent and a path.
- Run the rule checker and review which rule matched.
SEO TOOLS / FREE ONLINE TOOL
Robots.txt Tester
Review how a crawler may interpret Allow and Disallow rules for a URL path. Simple local rule testing; not an official Googlebot simulator.
✓ Browser only✓ No signup✓ Copy or download
Limitations and useful tips
This tester approximates common robots matching behavior and cannot replace verification with the official search engine crawler.
How to use Robots.txt Tester
- Paste a robots.txt document.
- Enter a crawler user-agent and a path.
- Run the rule checker and review which rule matched.
What to know before you start
This tester approximates common robots matching behavior and cannot replace verification with the official search engine crawler.