Robots.txt Generator

Free Robots.txt Generator – instantly generate custom robots.txt for SEO and crawling control.

Robots.txt Generator

Generate a clean robots.txt for SEO. Choose a preset, customize, then copy or download.

Preset

Sitemap URL(s) (one per line)

Crawl-delay (seconds, optional — Bing/Yandex only)

Block /wp-admin/ (Allow admin-ajax.php) Block /wp-includes/ Block search & query pages Block misc noise (/trackback/, /cgi-bin/) WooCommerce: block /cart/, /checkout/, /my-account/

Extra Disallow rules (one path per line)

Extra Allow rules (one path per line)

robots.txt

Notes & best practices

Google supports Allow/Disallow, wildcards (*) and end anchor ($), but ignores query-only rules. We use /*?s= pattern for search.
Crawl-delay is not used by Google; it’s honored by Bing and some others.
Keep robots.txt small and specific. For noindex, use meta tags; robots.txt only blocks crawling.