See also
Robots.txt Checker
Robots.txt Generator
Basic Allow All
Basic Block All
Advanced
1. Add User Agent Rules
All
Googlebot
Googlebot News
Googlebot Images
Googlebot Video
AdSense
AdsBot
BingBot
Yahoo!
DuckDuckGo
Baidu
Yandex
Applebot
Facebook
Twitter
Disallowed URLs
Disallow all
Allowed URLs
Allow all
You can add multiple blocks for different User Agents
Add User Agent Block
2. Add Sitemaps (optional)
Add Sitemaps
3. Generated Robots.txt
I am aware that I am using the robots.txt generator at my own risk. No liability will be accepted by this site for errors or indexing issues on the website.
Clear
Download
