Robots.txt & llms.txt Generator
Generate crawler rules that keep AI search bots allowed.
- Free
- No signup
- Runs in your browser
How to use Robots.txt & llms.txt Generator
- Enter your sitemap URL and any private paths to block, one per line.
- Choose whether to allow AI model training, then copy the robots.txt and upload it to your site root.
- Fill in your product name, a one-sentence summary, and your key pages.
- Copy the llms.txt and upload it to your site root, next to robots.txt.
Tip: Blocking AI training is separate from AI search. Keep AI search bots allowed if you want to be cited.
robots.txt
AI search bots (OAI-SearchBot, ChatGPT-User, PerplexityBot, Perplexity-User, Claude-SearchBot, Claude-User, Googlebot, Bingbot) always stay allowed, so you can appear in ChatGPT, Perplexity, Claude, and Google AI answers.
robots.txt
# Everyone, including Google, Bing, and AI search engines
User-agent: *
Allow: /
Disallow: /admin
Disallow: /api
llms.txt
llms.txt
# Your product
> One-sentence summary of what it is and who it is for.
Launched something?
List it on LaunchAF for a weekly launch and a permanent dofollow backlink.