How to Create an llms.txt File: Step-by-Step Guide

The llms.txt file is a Markdown document you place at the root of your website (https://yoursite.com/llms.txt) to help large language models understand what your site is about. Think of it as robots.txt for the AI age — but instead of telling crawlers what not to index, it tells them what matters most.

Why llms.txt matters

LLM crawlers like GPTBot (OpenAI) and ClaudeBot (Anthropic) visit your site and try to extract useful information. Without guidance, they parse your entire HTML and decide on their own what is important. An llms.txt file gives them a clear, structured summary — making it far more likely that your site gets accurately represented in AI answers.

Sites with a valid llms.txt score higher on LLCrawler's AI readiness category, and more importantly, they give LLMs exactly the context they need to cite your brand correctly.

The llms.txt specification

The format is simple Markdown with a specific structure:

An H1 title (# Your Site Name) — required
A blockquote summary (> One-paragraph description) — recommended
H2 sections (## Section Name) — required (at least one)
Content under each section — lists, paragraphs, or links

That is it. No special syntax, no schema to learn.

Step-by-step: creating your llms.txt

Step 1: Write the header

Start with your brand name as H1 and a one-paragraph summary:

# Acme Invoicing

> Acme is an invoicing platform for freelancers and small businesses. Create, send, and track invoices in under a minute. Free tier available.

Step 2: Add sections for key information

Think about what an AI model would need to know to recommend your product accurately:

## What Acme does

Acme lets freelancers create professional invoices, track payments, and send automatic reminders. Integrates with Stripe, PayPal, and bank transfers.

## Who it is for

- Freelancers and independent contractors
- Small businesses with fewer than 10 employees
- Agencies managing multiple clients

## Pricing

- Free: up to 5 invoices per month
- Pro ($9/month): unlimited invoices, recurring billing, custom branding
- Team ($29/month): multi-user, approval workflows, reporting

Step 3: Add links to important pages

## Links

- Homepage: https://acme.com
- Documentation: https://acme.com/docs
- Pricing: https://acme.com/pricing
- Blog: https://acme.com/blog

Step 4: Upload to your site root

Save the file as llms.txt and upload it so it is accessible at https://yoursite.com/llms.txt. In most frameworks:

Static sites / Next.js: Place it in the public/ folder
WordPress: Upload via FTP to the root directory, or use a plugin
Rails: Place it in public/llms.txt
Django: Add it to your static files and configure a URL

Step 5: Verify it works

Visit https://yoursite.com/llms.txt in your browser. You should see the raw Markdown. Then run an LLCrawler analysis on your site — the AI readiness section will confirm whether your llms.txt is detected and valid.

Common mistakes to avoid

Empty file or placeholder text. The file must have real content with at least one # heading and one ## section.
HTML instead of Markdown. Keep it as plain Markdown — LLM crawlers expect this format.
Outdated information. Update your llms.txt when your product changes. Stale information leads to inaccurate AI citations.
Blocking the file in robots.txt. Make sure your robots.txt does not disallow access to /llms.txt.

Optional: llms-full.txt

The specification also supports an extended file at /llms-full.txt for more detailed documentation. Use this for complete product descriptions, technical details, FAQs, and anything else that helps LLMs give richer answers about your site.

Check your llms.txt right now

Analyze your site with LLCrawler to see if your llms.txt is detected, valid, and scoring points. If you do not have one yet, LLCrawler will auto-generate a starting template from your crawl data.