robots.txt generator
Built in your browser · nothing is uploaded
Generates a robots.txt with disallow and allow rules and a sitemap reference. The critical thing to understand: robots.txt controls crawling, not indexing.
How to use the robots.txt generator
That distinction causes more damage than any other misunderstanding in technical SEO. Disallowing a page stops the crawler fetching it, but if other pages link to it, Google can still index the URL, showing it with no description because it was never allowed to read it. Worse, a page blocked in robots.txt cannot be seen to carry a noindex tag, so blocking a page you want removed actively prevents its removal. To keep something out of the index, allow crawling and use noindex. To keep something private, use authentication: robots.txt is a public file and a list of the directories you would rather people did not visit is an invitation to the curious. Note also that Google ignores crawl-delay entirely.
Questions
No. It stops crawling. A blocked URL linked from elsewhere can still appear in results, with no description.
Allow crawling and add a noindex tag. Blocking it in robots.txt prevents the noindex from ever being seen.
The opposite. It is public, and listing your admin paths tells anyone curious exactly where to look. Use authentication.
No, it ignores it entirely. Crawl rate is managed in Search Console.
The root of the domain, at /robots.txt. Nowhere else works.