tech

How to fix: Sitemap URL declared in robots.txt

Why this matters

Declaring the sitemap location inside robots.txt makes it discoverable by any crawler that reads robots — including ones that wouldn't otherwise guess the conventional /sitemap.xml path.

Background

robots.txt should declare your sitemap URL so crawlers find it without guessing. Even crawlers that check /sitemap.xml directly use this as confirmation.

References

sitemaps.org protocol §4 · robots.txt Sitemap directive

How to fix

Code snippet for each stack we cover. Pick the one matching your server / framework.

nginx

Edit robots.txt — add: Sitemap: https://your-site/sitemap.xml

apache

Same.

cloudflare

Same.

wordpress

Yoast / RankMath add this automatically when sitemap generation is enabled.

flask

Update your /robots.txt route to include the Sitemap line.

express

Same.

rails

Same.

Verify it's working

curl -s https://your-site/robots.txt | grep -i sitemap — should show your full sitemap URL.

Want to know if your site has this issue?

Run a free 53-check audit — security, GDPR, NIS2, and technical SEO.

Audit my site →