Robots.txt Tester

Validate your robots.txt and test URL access rules for different crawlers

robots.txt

14 lines
1
2
3
4
5
6
7
8
9
10
11
12
13
14

Parsed Rules

*
Disallow: /admin/
Disallow: /api/
Disallow: /private/
Allow: /api/public/
Googlebot
Allow: /
Bingbot
Allow: /
Disallow: /internal/

Sitemaps

https://optimised.marketing/sitemap.xml

Quick Templates

Need help with technical SEO?

Our team can audit your entire technical SEO setup including robots.txt, sitemaps, and crawl optimisation.

Explore SEO Services