Robots.txt Generator
Generate a robots.txt file to control how search engines crawl your website.
Rule 1
User-agent: * Allow: / Disallow: /admin/ Disallow: /private/ Disallow: /api/ Sitemap: https://example.com/sitemap.xml
Generate a robots.txt file to control how search engines crawl your website.
User-agent: * Allow: / Disallow: /admin/ Disallow: /private/ Disallow: /api/ Sitemap: https://example.com/sitemap.xml
Recommended Tools & Services