What is llms.txt? Format, examples and whether it matters

llms.txt is a plain Markdown file published at yoursite.com/llms.txt that gives large language models and AI agents a short, curated overview of a website: what it is, plus links to the pages worth reading. Jeremy Howard of Answer.AI proposed it in September 2024 at llmstxt.org, and a revised v2 of the proposal was published in August 2026. It is a community proposal, not a web standard. Google has said publicly that you don’t need it to appear in AI Overviews, so treat it as a cheap help for agents, not a ranking lever.

What is llms.txt, in one paragraph?

Web pages are built for people: navigation, cookie banners, scripts and layout wrap the actual information. An agent that wants to answer a question about your product has to fetch HTML, strip it, and guess which pages matter, all while paying for every token. llms.txt solves the “which pages matter” part. It sits at a predictable URL (like robots.txt and sitemap.xml), uses Markdown so both humans and models can read it, and follows a fixed structure so ordinary parsers can process it too. The proposal’s stated expectation is that agents read or search the llms.txt file, then follow the links that are relevant to the task.

The proposal also recommends that pages agents might need offer a clean Markdown version at the same URL with .md appended (page.html.md) or the extension replaced (page.md).

What is the exact llms.txt format?

The llmstxt.org spec defines these sections, in this order:

By convention, an H2 called ## Optional holds secondary links an agent can skip when it needs a shorter context. In v1 that section had a mechanical meaning for context-expansion tools; v2 dropped that and keeps it only as a convention.

Location rules from v2:

• The file can live at the root (/llms.txt) or any subpath (/docs/llms.txt).
• A file covers the URLs under its path. When more than one file applies, agents should use the most specific one.
• To help agents find things, pages can point to their Markdown version with rel="alternate" type="text/markdown" and to the covering llms.txt with rel="describedby", either as HTML <link> elements or an HTTP Link: header.

What does a full llms.txt example look like?

Here is a complete file for a hypothetical B2B SaaS company. It follows the spec exactly: one H1, one blockquote, a short notes list with no headings, then H2 file lists.

# Acme Analytics

> Acme Analytics is a product analytics platform for B2B SaaS teams. It tracks events from web and mobile apps, builds funnels and retention reports, and syncs audiences to HubSpot and Salesforce. Pricing is per tracked user, with a free tier up to 10,000 monthly tracked users.

Important notes:

- Acme is not a web analytics tool for anonymous traffic; it requires an identified user ID on each event.
- The REST API is versioned; v3 is current and v2 is deprecated.
- Self-hosting is not offered. Data is stored in the US or EU region, chosen at signup.

## Product

- [Product overview](https://acme.example/product.md): What Acme does, who it is for, core features
- [Pricing](https://acme.example/pricing.md): Plans, tracked-user limits, what counts as a tracked user
- [Security and compliance](https://acme.example/security.md): SOC 2 report, GDPR, data residency

## Docs

- [Quickstart](https://acme.example/docs/quickstart.md): Install the SDK and send a first event in 10 minutes
- [Event tracking reference](https://acme.example/docs/events.md): Event schema, properties, naming rules
- [REST API v3](https://acme.example/docs/api-v3.md): Endpoints, authentication, rate limits

## Comparisons

- [Acme vs. Mixpanel](https://acme.example/compare/mixpanel.md): Where each fits and how to migrate

## Optional

- [Changelog](https://acme.example/changelog.md): Release notes by month
- [Blog](https://acme.example/blog.md): Product and analytics articles

For a real-world reference, the llmstxt.org page uses a cut-down version of the FastHTML project’s file, which lives at fastht.ml/docs/llms.txt and covers only the docs path.

Writing tips that follow from the spec:

• Keep the file small enough to fit in a context window; the detail belongs behind the links.
• Put the facts an agent most often gets wrong in the blockquote and notes (pricing model, what you are not, deprecated versions).
• Write link notes that say what the page answers, not marketing copy.
• Test it the way the spec suggests: give an AI assistant only your llms.txt and ask it questions about your product.

What is llms-full.txt?

llms-full.txt is a single file that concatenates the full Markdown content of a site (usually a documentation site) so an agent can load everything in one request. It is not part of the llmstxt.org specification. It is a convention popularized by documentation platforms: Mintlify, for example, auto-generates both /llms.txt (an index of pages with short descriptions) and /llms-full.txt (every page’s title, URL, description and full Markdown content) for the docs sites it hosts.

If you run a marketing site with a blog, you rarely need llms-full.txt. If you run developer docs, it can be genuinely useful for people who paste your docs into an IDE assistant.

Which AI companies use llms.txt?

This is where most articles overstate things. Here is what is actually documented as of October 2026:

Some third-party write-ups claim a given AI vendor “confirmed support.” Unless you can find the statement in the vendor’s own docs, treat it as unverified. You may see AI crawlers request /llms.txt in your server logs; a fetch shows the file was read, not that it influenced an answer.

The usage pattern that is real: agents and coding assistants fetch an llms.txt when a user or tool points them at it, especially for software documentation. The llmstxt.org v2 text says files are “used most heavily for software documentation, where coding agents follow them to find API references and tutorials.”

How is llms.txt different from robots.txt and sitemap.xml?

All three live at predictable URLs, but they answer different questions.

Two practical consequences:

1. llms.txt cannot block anything. If you want to keep content out of AI training or AI search, use robots.txt rules for the relevant user agents (for example GPTBot, OAI-SearchBot, ClaudeBot, Google-Extended). See GEO technical implementation for the current bot list.
2. llms.txt does not replace your sitemap. The spec itself explains why a sitemap is not a substitute: it often lacks Markdown versions, excludes useful external URLs, and is usually too large for a context window. The reverse is also true: search engines still need the sitemap.

How do you create an llms.txt file, step by step?

Budget 30 to 60 minutes for a typical marketing site.

1. Pick 10 to 30 pages. Product overview, pricing, top use cases, comparison pages, key docs, security/compliance, contact. Skip thin pages and individual blog posts unless they are cornerstone content.
2. Write the H1 and blockquote. One to three sentences: what you are, who it is for, the pricing model, and anything commonly misunderstood. This is the most valuable part of the file.
3. Add notes. A short list of facts that prevent wrong answers (“not available in X,” “v2 API is deprecated”). No headings in this block.
4. Group links under H2s. Product, Docs, Use cases, Comparisons, Company, Optional. Each item: - [Title](URL): what this page answers.
5. Link to Markdown versions if you have them. If not, link to the HTML pages; the spec recommends Markdown but does not require it.
6. Save as UTF-8 plain text and serve it at /llms.txt with a text/plain or text/markdown content type and a 200 status.
7. Validate. Open the URL in a browser, confirm there is exactly one H1, and run Lighthouse to make sure the agentic browsing check does not report a server error.
8. Keep it current. Add it to the release checklist for pricing, product and docs changes. A stale llms.txt that contradicts your pricing page is worse than none.

Creating llms.txt for WordPress, docs platforms and site builders

• WordPress: Yoast SEO and AIOSEO both offer an llms.txt generator.
• Docs platforms: Mintlify and GitBook serve one automatically for hosted docs.
• Wix: generates one for every site.

Generated files list many pages by default. Review and trim them; curation is the point.

Creating llms.txt for an Astro or other static site

The simplest option works on any static site generator (Astro, Hugo, Eleventy, Next.js static export): write the file by hand and drop it in the folder that is copied verbatim to the site root. In Astro that is public/:

public/
  robots.txt
  llms.txt      <- served at /llms.txt

If you want it to stay in sync with your content, generate it at build time. In Astro, a static endpoint in src/pages/llms.txt.ts is prerendered to /llms.txt during the build. This example builds the file from a guides content collection:

// src/pages/llms.txt.ts
import type { APIRoute } from 'astro';
import { getCollection } from 'astro:content';

export const GET: APIRoute = async ({ site }) => {
  const guides = (await getCollection('guides', ({ data }) => !data.draft))
    .sort((a, b) => a.data.title.localeCompare(b.data.title));

  const base = site?.toString().replace(/\/$/, '') ?? '';
  const lines = [
    '# Example Co',
    '',
    '> Example Co is a go-to-market platform for B2B teams. One-sentence pricing and audience summary here.',
    '',
    '## Product',
    '',
    `- [Platform overview](${base}/platform/): What the product does and which channels it covers`,
    '',
    '## Guides',
    '',
    ...guides.map((g) => `- [${g.data.title}](${base}/guides/${g.id}/): ${g.data.description}`),
    '',
  ];

  return new Response(lines.join('\n'), {
    headers: { 'Content-Type': 'text/plain; charset=utf-8' },
  });
};

Notes for this approach:

• Set site in astro.config.mjs so absolute URLs are correct.
• Do not also keep a public/llms.txt, or the two will collide.
• To offer Markdown versions of pages, add a parallel endpoint (for example src/pages/guides/[slug].md.ts) that returns each entry’s raw body with a text/markdown content type, then point the llms.txt links at those .md URLs.

Should you bother with llms.txt?

Short answer: yes if you have documentation or an API, optional for everyone else, and never at the expense of real GEO work.

What actually moves AI visibility, in rough priority order:

1. Allow the retrieval bots you want citations from in robots.txt, and serve key content in server-rendered HTML.
2. Answer the query directly at the top of each section, with sourced facts and numbers. See the AEO / GEO content optimization guide.
3. Earn mentions on the third-party pages AI answers already cite (reviews, listicles, communities). See the GEO playbook.
4. Measure whether you are mentioned and cited for your target prompts. See how to check your LLM SEO visibility.

llms.txt sits after all of these. It is a tidy, cheap signal of intent that some agents will use, and it costs little to maintain if you generate it from your content.

Frequently asked questions about llms.txt

Is llms.txt an official standard?

No. It is a community proposal hosted at llmstxt.org with a public GitHub repository for discussion. robots.txt, by contrast, is RFC 9309.

Does Google use llms.txt?

Not for Search. Google’s AI features documentation says you don’t need new machine-readable files or AI text files to appear in AI Overviews or AI Mode. Separately, Chrome’s Lighthouse includes an optional llms.txt check as part of agentic browsing audits.

Can llms.txt stop AI companies from training on my content?

No. It has no access-control semantics. Use robots.txt directives for the specific AI user agents, and check each vendor’s crawler documentation.

Where should the file go if my docs are on a subpath?

v2 of the proposal allows /docs/llms.txt; it covers everything under /docs/, and the most specific file wins when several apply.

Do I need Markdown versions of every page?

No. The spec recommends them for pages agents might need, but an llms.txt that links to normal HTML pages is still valid.