tech

How to fix: Sitemap URL declared in robots.txt

Part of the Technical SEO pillar on the Site Integrity Index.

Why this matters

Declaring the sitemap location inside robots.txt makes it discoverable by any crawler that reads robots — including ones that wouldn't otherwise guess the conventional /sitemap.xml path.

Background

robots.txt should declare your sitemap URL so crawlers find it without guessing. Even crawlers that check /sitemap.xml directly use this as confirmation.

References

sitemaps.org protocol §4 · robots.txt Sitemap directive

How to fix

Code snippet for each stack we cover. Pick the one matching your server / framework.

nginx

Edit robots.txt — add: Sitemap: https://your-site/sitemap.xml

apache

Same.

cloudflare

Same.

wordpress

Yoast / RankMath add this automatically when sitemap generation is enabled.

flask

Update your /robots.txt route to include the Sitemap line.

express

Same.

rails

Same.

Verify it's working

curl -s https://your-site/robots.txt | grep -i sitemap — should show your full sitemap URL.

Want to know if your site has this issue?

Run a free audit — security, GDPR, technical SEO, AEO/GEO, and WCAG accessibility.

Audit my site →