If you have been following the AI search conversation lately, you have probably seen the term llms.txt come up. Someone in a Slack group posts it. A consultant adds it to a checklist. A thread on X calls it the next big thing for SEO. And then someone else replies that it is completely pointless and no one uses it. Both reactions are a little right and a little wrong.
Here is the straightforward version: llms.txt is a community-proposed plain-text file placed at your site root that summarizes the site and links selected pages. Some tools may use the convention, but Google says it ignores the file for Search and its generative AI features.
Think of it as leaving a well-organized briefing packet on your front desk, instead of making a visitor wander through every room to figure out what you do.
Where the idea comes from
The spec was proposed by Jeremy Howard (of fast.ai and Answer.AI) at llmstxt.org. The core insight is simple: HTML is built for browsers. It is full of navigation menus, cookie banners, JavaScript, ads, and boilerplate that adds noise for a model trying to understand what a page is actually about. A clean markdown-formatted plain-text file strips all of that away.
The proposal is not an official standard from any standards body. It is a community proposal that some site owners have started adopting. Browser-vendor support, crawler support, and AI engine support are all patchy and evolving. That context matters when you are deciding how much energy to spend on it.
What the file actually looks like
The spec is refreshingly simple. The file uses markdown conventions. It starts with an H1 containing your site or brand name, followed by a blockquote with a one-sentence summary of what you do. After that, you add H2 sections with markdown-formatted links to your most important pages, with a short description for each link.
llms.txt (example for a fictional analytics product)
# Acme Analytics > Privacy-first web analytics for small SaaS teams. Self-serve, no cookies required. ## Product - [Features](https://acme.com/features): what Acme tracks and how the dashboard works - [Pricing](https://acme.com/pricing): plans, limits, and the free tier - [Changelog](https://acme.com/changelog): recent releases and what changed ## Docs - [Quick Start](https://acme.com/docs/quick-start): install the snippet and verify your first event - [API Reference](https://acme.com/docs/api): REST API endpoints and authentication - [Privacy FAQ](https://acme.com/docs/privacy): data retention, GDPR, and cookie-free tracking ## Company - [About](https://acme.com/about): the team and why we built this - [Blog](https://acme.com/blog): product updates and analytics thinking ## Contact - Website: https://acme.com - Support: support@acme.com
That is it. There is no schema to validate, no JSON to format, no plugin to install. If you can edit a text file and upload it to your server root, you can ship an llms.txt in under fifteen minutes.
How it differs from robots.txt and sitemap.xml
People often ask how llms.txt relates to the existing files you probably already have. Here is a quick comparison.
| File | Primary purpose | Who reads it | What it controls |
|---|---|---|---|
| robots.txt | Access control for crawlers | Search engine bots, any web crawler | Which pages a crawler is allowed or disallowed from fetching |
| sitemap.xml | Page discovery | Search engine crawlers | A structured list of all URLs on your site and their metadata |
| llms.txt | Meaning and context | AI models and LLM-powered tools (in theory) | What your site is about and which pages carry the most important information |
They serve different needs and are not substitutes for each other. robots.txt is about permissions. sitemap.xml is about completeness. llms.txt is about interpretation. You can have all three without any conflict.
The honest debate: does it actually matter?
Here is where I want to be straight with you, because a lot of content on this topic skips past the real uncertainty.
The case for llms.txt
- ✓It is low effort. Writing one takes less time than a single blog post.
- ✓It costs nothing to publish. No tools required, no ongoing maintenance unless your site structure changes.
- ✓It can serve as a maintained plain-text index for tools or people that intentionally choose to read it.
- ✓It is a low-risk experiment when it is generated from current facts and does not displace higher-value work.
- ✓Writing the summary can expose unclear product language, even when the file itself has no search effect.
The case for skepticism
- ✓Google explicitly says it ignores llms.txt for Search and generative AI features.
- ✓The spec is not endorsed by any standards body. Adoption is voluntary and inconsistent.
- ✓LLMs are trained on large web crawls. A single text file at your root is a tiny signal compared to thousands of pages of indexed content.
- ✓If your pages are already well-structured with clean headings, good metadata, and authoritative content, you may already be doing most of what llms.txt would add.
The file is optional. Ship it only when the maintenance cost is genuinely small, and never put it ahead of useful pages, internal links, access, or evidence.
Who should bother writing one?
Short answer: most sites can skip it without a Google Search penalty. It is most defensible for teams intentionally testing the convention and willing to keep the summary current.
- SaaS products and tools. If someone asks an AI assistant for the best privacy-friendly analytics tool and you want to be in the answer, having a clean machine-readable summary of what you do is a reasonable hedge.
- Consultants, agencies, and solo operators with a clear service offering. Your site might be ten pages, which makes the file trivial to write and arguably more useful since you can cover your whole site in one tight summary.
- Publishers and content-heavy sites. Use the file to highlight your most important categories and cornerstone pieces rather than a full index.
- Sites that are already investing in AEO or GEO. If you are thinking about how AI systems find and cite your content, llms.txt is a natural part of that checklist.
Who can probably skip it for now: sites whose primary audience is not searching for their type of content via AI chat interfaces, or teams that have ten more important things to fix first (page speed, mobile experience, thin content). Get the fundamentals right before worrying about emerging signals.
Tips for writing a good one
- ✓Write the blockquote summary as if you were answering the question what is this site, in one sentence, for a smart stranger. Clear and specific beats clever.
- ✓Link to pages that actually explain what you do, not just your homepage. Docs pages, feature pages, and your pricing page are often more useful than your blog index.
- ✓Keep descriptions on each link short and specific. Plans and pricing tells a model more than learn more.
- ✓Aim for maybe ten to twenty links total. A 200-link dump of every URL on your site defeats the purpose of the summary.
- ✓Update it when your site structure changes, like when you add a major product or restructure your docs.
The bigger picture: being machine-legible
The broader machine-legibility work belongs on the actual pages: clear headings, visible facts, accessible links, current evidence, consistent entities, and structured data that matches visible content. llms.txt does not substitute for any of it.
Want a working llms.txt in under two minutes? The free generator tool below analyzes your site and drafts the file for you, so you can review and ship it rather than write it from scratch.
Optional
Google Search does not require or use llms.txt, so treat the file as an experiment rather than an AEO score item
The bottom line
llms.txt is an experimental publishing convention, not a search signal you can defend from current Google guidance. If a tool you use consumes it, keep the file honest and current. Otherwise, skipping it is a reasonable choice.
Spend the real energy on helpful page content, internal discovery, factual consistency, primary evidence, and the independent sources buyers actually use.