ToolEmpire

robots.txt Generator

Create a clean robots.txt with disallow rules, crawl delay, and sitemap reference.

Inputs

Results

User-agent: *

About this tool

Enter the paths to block, a crawl delay, and your sitemap URL to get a ready-to-upload robots.txt. Rules are deduplicated and paths get leading slashes.

How to use

  1. 1List the paths to block, one per line, e.g. /admin.
  2. 2Set a crawl delay if needed (0 leaves it out) and add your sitemap URL.
  3. 3Save the output as robots.txt at your site root and upload it.

Frequently asked questions

Related tools

Duplicate Content CheckerCompare two texts and see exactly how similar they are.Keyword Difficulty CheckerEstimate how hard a keyword is to rank for.SEO Audit ToolScan a page for on-page SEO issues.Schema Markup GeneratorGenerate JSON-LD structured data for any page type.Domain Rating CheckerCheck a domain's authority and backlink strength.Meta Title GeneratorCreate SEO-optimized meta titles with the right length.