llms.txt Explained: What It Is, Who Reads It, and How to Write One

What llms.txt Actually Is

llms.txt is a plain markdown file you place at the root of your domain — https://example.com/llms.txt — that describes your site in a form a language model can read in one pass.

That's the whole idea. A model or an AI agent that lands on your domain normally has to crawl page after page, strip navigation and scripts, and infer what your business does from fragments. An llms.txt file gives it a short, curated summary plus a list of the pages that actually matter, written in markdown instead of buried in markup.

The convention was proposed by Jeremy Howard of Answer.AI in late 2024 and has since been adopted voluntarily by a growing number of documentation sites, SaaS products, and developer tools.

What It Is Not

Being precise here matters more than hype, so let's be blunt about the limits:

• It is not a ranking signal. No AI provider has published a statement saying llms.txt improves your chances of being cited.
• It is not an access-control file. That's robots.txt. llms.txt doesn't allow or block anything.
• It is not universally consumed. Support is inconsistent and largely undocumented across providers. Treat it as a low-cost, forward-looking convention, not a guaranteed pipeline into ChatGPT.
• It will not fix a site an AI can't read. If your content only renders after JavaScript hydration, or your headings are <div>s, llms.txt won't rescue you.

Anyone telling you llms.txt is the secret to getting recommended by ChatGPT is selling something. What it genuinely does is reduce ambiguity about who you are — which is useful whether or not a specific crawler parses it today.

llms.txt vs robots.txt vs sitemap.xml

These three files do genuinely different jobs, and having one doesn't replace another:

| File | Audience | Job | | --- | --- | --- | | robots.txt | All crawlers | Permission — what may be fetched | | sitemap.xml | Search crawlers | Inventory — every URL worth indexing | | llms.txt | Language models | Orientation — what this site is and what to read first |

A well-set-up site has all three. robots.txt should explicitly allow the AI user-agents you want reading you (GPTBot, ClaudeBot, PerplexityBot, Google-Extended). sitemap.xml should list every canonical URL. llms.txt should be short and opinionated — the ten links you'd hand a new employee, not all four hundred.

The Format

There's no formal spec with validators and error codes. The convention that has settled in practice is:

1. An H1 with the site or product name.
2. A blockquote with a one- or two-sentence summary.
3. Optional paragraphs of extra context.
4. H2 sections containing markdown link lists, each link optionally followed by a colon and a short description.
5. An optional ## Optional section for lower-priority links a model can skip if it's short on context.

A working template

# Acme Analytics

> Acme Analytics is a self-hosted product analytics platform for
> engineering teams who need event data without sending it to a
> third party.

Acme is open source (Apache 2.0), runs on Postgres, and is used
primarily by B2B SaaS teams with compliance requirements.

## Core documentation

- [Quickstart](https://acme.com/docs/quickstart): install and send your first event in 5 minutes
- [Self-hosting guide](https://acme.com/docs/self-host): Docker and Kubernetes deployment
- [Event schema](https://acme.com/docs/schema): the data model and reserved properties
- [API reference](https://acme.com/docs/api): REST and ingestion endpoints

## Product

- [Pricing](https://acme.com/pricing): free self-hosted tier, cloud from $49/mo
- [Comparison vs hosted analytics](https://acme.com/compare): where Acme fits

## Optional

- [Changelog](https://acme.com/changelog)
- [Engineering blog](https://acme.com/blog)

Six Rules That Make the Difference

Most llms.txt files we see in the wild are near-useless because they're either a bare title or a dump of every URL on the site. The ones that work follow the same handful of rules:

1. Lead with the definition, not the pitch

The blockquote should answer "what is this?" in a way a stranger could repeat back correctly. "The future of team productivity" tells a model nothing. "A Kanban task manager for construction site foremen, with offline mobile support" tells it everything.

2. Use absolute URLs

Relative paths are ambiguous once the file is pulled out of context. Always write the full https:// URL.

3. Describe every link

A bare link is a guess. [Pricing](https://acme.com/pricing): free self-hosted tier, cloud from $49/mo embeds the answer to a question a user might ask, right in the index.

4. Keep it under about 50 links

This is an orientation document, not a sitemap. If a link wouldn't be in your top-50 most useful pages, it belongs in sitemap.xml instead.

5. Serve it as text/plain and don't redirect it

A surprising number of llms.txt files return HTML, a 404 page with a 200 status, or a redirect chain into a docs app. Check the raw response, not what the browser renders.

6. Keep it in sync with your site

A file listing pages you deleted six months ago is worse than no file. If your content changes often, generate it in your build.

The Bigger Version: llms-full.txt

Some sites also publish llms-full.txt — the same idea, but with the actual content of the key pages inlined as markdown rather than just linked. This is most common on documentation sites where the goal is for a model to be able to answer usage questions without following any links.

It's a genuine trade-off. llms-full.txt can run to hundreds of kilobytes, which is a lot of context to spend, and it gets stale much faster. Start with llms.txt; add the full version only if your primary use case is technical documentation and you can generate it automatically.

How to Check Yours

Three quick checks, in order:

1. Does it exist and return plain text? curl -sI https://yoursite.com/llms.txt — you want a 200 and a text/plain content type.
2. Does every link resolve? Dead links in an orientation file actively mislead.
3. Does the summary match the site? Read the blockquote cold and ask whether it describes what a visitor would actually find.

Then look at the layer underneath. llms.txt is one check inside a much bigger picture: whether your pages have clean heading hierarchies, valid structured data, visible publication dates, defined key terms, and robots.txt rules that let AI crawlers in at all. A perfect llms.txt on a site that blocks GPTBot accomplishes nothing.

Where This Fits in a GEO Strategy

Honest ordering of effort, highest impact first:

1. Let the crawlers in. Audit robots.txt for the AI user-agents.
2. Make content machine-readable. Server-rendered HTML, semantic structure, single H1, real headings.
3. Add structured data. Organization, Product, Article, FAQPage — whatever genuinely applies.
4. Define your entities. Say clearly what you are, who it's for, and what it costs, in text.
5. Publish llms.txt. Cheap, fast, and it removes ambiguity.

llms.txt sits at the end of that list on purpose. It's a twenty-minute task with a real but modest upside — worth doing, not worth doing first.

Check Your Site

LLMao's free analyzer checks llms.txt discovery alongside 34 other AI-readability tests, including crawler access rules, structured data validity, heading structure, and entity clarity. Most of those checks are deterministic, so you can fix something, rescan, and see exactly what moved.

Run your domain and see where llms.txt sits relative to the things that matter more.