Robots.txt & XML Sitemap Helper
Create a safe robots.txt file and a basic XML sitemap for your website, or paste an existing sitemap to check it for common mistakes.
Free — no sign-up needed
Common rules
Rules
No rules — every crawler may visit every page (the safe default for most business sites).
robots.txt
User-agent: * Disallow:
Upload it to the root of your site so it opens at /robots.txt. robots.txt is a request to well-behaved crawlers: it does not remove pages from search results and does not protect private pages — use passwords for those, and a noindex tag to keep a page out of results.
0 valid URL(s)
sitemap.xml
Add at least one valid URL.
A basic helper: it only includes the URLs you enter — it doesn't crawl your site. Upload the file to your site's root and submit it in Google Search Console. FADRO can't confirm that Google has received or indexed it.
Works without an account
- Generate robots.txt with common presets
- Build a sitemap from a list of URLs
- Validate pasted sitemap XML
- Copy or download the files
With a free account
- Save robots.txt and sitemap drafts
- Reopen and update them as your site grows
Optional and free. Basic results never require registration.
How to use it
- Enter your website address. Add rules only for areas crawlers shouldn't visit, such as admin or cart pages.
- Download robots.txt and upload it to your site's root so it opens at yoursite.com/robots.txt.
- Paste your public page URLs into the sitemap generator, download sitemap.xml and upload it to the root too.
- Submit the sitemap address in Google Search Console.
What these files do
robots.txt tells well-behaved crawlers which parts of a site they shouldn't visit. It is not security — anyone can still open a listed page — and it doesn't remove pages from search results. Use password protection for private pages and a noindex tag for pages you want kept out of results.
An XML sitemap lists the public pages you'd like search engines to know about. It helps discovery, especially for new sites, but doesn't guarantee indexing.
This helper works entirely from what you type or paste — it doesn't fetch or crawl your website. To see whether your live robots.txt and sitemap are reachable, run the free website audit.