The llms.txt guide: what it is, and what it honestly does
llms.txt is a proposed convention: one markdown file at your site's root that gives language models a curated map of your content. Here is the exact format, a worked example, who actually supports it today, and where it fits in a serious GEO strategy.
One plan, $199/month. No contract, cancel anytime.
llms.txt is a markdown file served at /llms.txt that gives large language models a curated map of your site: what it is, what matters, and where to read more. It was proposed by Jeremy Howard of Answer.AI in September 2024, and it is a convention, not a standard any engine has committed to.
That last clause matters, and most articles about llms.txt omit it. This guide covers the format precisely, then is honest about adoption, because you should know what you are buying with the hour it takes to write one.
The idea: a site map written for models, not crawlers
Web pages are noisy for a language model: navigation, scripts, banners, and boilerplate surround the content, and context windows are finite. The llms.txt proposal answers that with a single, clean markdown file at your root that a model, or a tool acting for one, can fetch to understand your site at a glance. Curation is the feature. Where a sitemap enumerates everything, llms.txt says: here is who we are, and here are the pages that actually matter, each with a line about why.
The format, exactly
The proposed spec is minimal and readable. One H1 with the site name, a blockquote summary, optional context paragraphs, then H2 sections containing link lists, each link followed by a short description. An Optional section, by convention, marks links a model can skip when context is tight. A worked example:
# SmashSERP > SmashSERP is an AI SEO and GEO platform. It does keyword research, > writes answer-first articles, publishes to WordPress, tracks rankings, > and verifies whether ChatGPT, Perplexity, Gemini, and Google AI > Overviews cite your site. ## Product - [Features](https://smashserp.online/features): What the platform does, end to end - [Pricing](https://smashserp.online/pricing): One plan, $199/month, no contract ## Guides - [What is GEO?](https://smashserp.online/what-is-geo): Generative engine optimization explained - [Answer engine optimization](https://smashserp.online/answer-engine-optimization): How to be the answer, not just a link ## Optional - [Blog](https://smashserp.online/blog): All articles
Serve it as plain text or markdown at the root of your domain. The companion convention, llms-full.txt, goes further and inlines the full content of your key pages into one file, most useful for documentation sites that want tools to ingest everything in a single fetch. Some sites also publish markdown twins of individual pages by appending .md to the URL, a related practice from the same proposal.
llms.txt vs robots.txt vs sitemap.xml
The three root files are often conflated, and they do three different jobs:
- robots.txt is permission: which crawlers may fetch which paths. It is where you allow OAI-SearchBot or PerplexityBot, as covered in our ChatGPT citation guide.
- sitemap.xml is discovery: an exhaustive machine list of URLs for search crawlers, with no opinion about importance beyond hints.
- llms.txt is curation: a human-written brief telling a model what your site is and which pages carry the substance.
None substitutes for another. A model blocked by robots.txt will not be saved by a beautiful llms.txt, and llms.txt describes meaning in a way neither of the older files attempts.
Honest status: who actually reads it
Now the part that separates observed fact from hope. Observed: adoption is real and growing among developer documentation platforms and AI-focused companies, and various AI tools and frameworks will fetch an llms.txt when pointed at a site. Also observed: no major AI engine, not OpenAI, Google, Anthropic, or Perplexity, has publicly confirmed using llms.txt in retrieval or training, and Google's John Mueller has likened it to the keywords meta tag, a signal sites want to send rather than one engines are known to consume.
So the honest framing is a cheap option, not a tactic. It costs an hour, it cannot hurt, it is genuinely useful to the tools that do read it, and if a major engine adopts the convention you are already positioned. What it is not, on current evidence, is a lever that moves citations this quarter. Anyone selling it as one is ahead of the facts.
How to write yours
- List the pages you would show a smart new employee on day one: product, pricing, docs, and your best guides. A few dozen links at most.
- Write one plain sentence per link saying what a reader learns there. No marketing copy, models do not convert.
- Open with a blockquote summary that states what you are and who you serve in two or three sentences, the same entity description you use everywhere else.
- Serve it at /llms.txt, keep it current when key pages change, and consider llms-full.txt only if you run documentation worth ingesting whole.
Where it fits in a GEO strategy
Keep proportions honest: llms.txt is a small piece of answer engine optimization. The levers with observable effect on whether AI engines cite you are answer-first structure, self-contained passages, entity clarity, schema, crawler access, and authority, the full playbook in how to get cited by AI. Do those first, add llms.txt as the cheap finishing touch, then verify instead of assuming: SmashSERP checks whether ChatGPT, Perplexity, Gemini, and Google AI Overviews cite your site and captures the exact passages they quote, so your AI visibility is a measured number, not a hope.
llms.txt FAQ
What is llms.txt?
llms.txt is a proposed convention: a markdown file served at your site's root, at /llms.txt, that gives large language models a curated, readable map of your most important content. It contains your site's name, a one-paragraph summary, and organized lists of links with short descriptions. It was proposed by Jeremy Howard of Answer.AI in September 2024 and has since been adopted mostly by developer documentation sites.
Is llms.txt the same as robots.txt?
No, they do opposite jobs. robots.txt tells crawlers what they may not access, a permission file. llms.txt tells language models what they should read first and what it means, a curation file. One restricts, the other recommends. A site can sensibly have both, and neither replaces the other or your XML sitemap.
Do ChatGPT, Google, or Perplexity actually read llms.txt?
As of this writing, no major AI provider has publicly committed to using llms.txt in its retrieval or training pipelines, and Google's John Mueller has publicly compared it to the old keywords meta tag, useful mainly as a signal site owners want to send. Some AI-adjacent tools and frameworks fetch it, and adoption is real among documentation platforms, but you should treat engine support as unconfirmed, not assumed.
What is llms-full.txt?
A companion convention: where llms.txt is a curated index of links, llms-full.txt inlines the full text of your important pages into one large markdown file, so a model or tool can ingest your content in a single fetch without crawling. It is most common for software documentation, where tools load an entire docs set into a model's context window at once.
Will llms.txt improve my AI citations?
There is no evidence it directly increases citations today, and we will not pretend otherwise. The levers with observable effect are answer-first structure, entity clarity, schema, crawlability, and authority. llms.txt is a cheap, low-risk addition on top of those: it takes an hour, cannot hurt, and positions you if adoption grows. It is a reasonable bet, not a ranking tactic.
How big should an llms.txt file be?
Small and curated, that is the point. Link your genuinely important pages, a few dozen at most for a typical site, each with a one-line description that says what a reader learns there. Dumping your entire sitemap into it defeats the purpose, which is to give a model your site's shape at a glance. Curate like you are briefing a new employee in one page.
Does SmashSERP help with llms.txt?
SmashSERP focuses on the levers with measurable effect: it writes answer-first articles with FAQ schema, runs technical audits, and verifies whether ChatGPT, Perplexity, Gemini, and Google AI Overviews actually cite your site, capturing the exact quoted passages. That verification is how you find out what your AI visibility responds to, llms.txt included, instead of guessing.
Keep exploring SmashSERP
Optimize the levers that move citations.
SmashSERP structures content for AI answers and verifies whether ChatGPT, Perplexity, Gemini, and AI Overviews cite you, exact passages included. $199/month, cancel anytime.
One plan, $199/month. No contract, cancel anytime.