As of 4 October 2026, Adobe has shipped tools that measure and improve how AI assistants read an...
What is llms.txt, and should your website have one?
By Rishabh Gupta, founder of ROAS Expert. Last updated 19 October 2026.
llms.txt is a plain Markdown file you put at the root of your site (yoursite.com/llms.txt) that tells AI assistants what your site is, what matters on it, and where the clean versions of your key pages live. It was proposed by Jeremy Howard in September 2024 and it is a convention, not a standard anyone is forced to follow. Should you have one? Yes if it takes you 30 minutes and you're on a platform that can serve it. No if you expect it to move rankings or get you recommended by ChatGPT on its own, because there's little evidence it does either.
What llms.txt actually is
The proposal lives at llmstxt.org. Its own summary is "a proposal to standardise on using an /llms.txt file to provide information to help agents use a website." The idea is simple. Language models have small context windows compared to the size of a website, and HTML is full of nav, scripts and boilerplate they don't need. A short curated index in Markdown gives an agent the important bits without the noise.
The spec says the file is meant mainly for use at inference time, when an assistant is answering a question or completing a task, rather than for training. That distinction matters later when we talk about who reads it.
The format
The file is Markdown with a fixed order of parts:
- An H1 with the site or project name. This is the only required element.
- A blockquote with a one or two sentence summary of the site.
- Optional paragraphs or lists with more detail, but no further headings here.
- Zero or more H2 sections, each a list of links in the form
[name](url): short note. - An H2 called "Optional" for secondary links an agent can skip when it is short on context.
Many sites also publish llms-full.txt. The spec page doesn't define it, but docs hosts do. Mintlify's documentation describes it as a single file that combines the entire docs site as context, with each page's title, source URL, description and full Markdown content. Think of llms.txt as the table of contents and llms-full.txt as the whole book.
Who has adopted it
Adoption is real but concentrated in developer documentation. I checked these myself on 19 October 2026:
- Anthropic serves one at platform.claude.com/llms.txt, with an H1 "Anthropic Developer Documentation", sections for Messages, Admin, SDKs and the API reference, and a pointer to llms-full.txt at the end.
- Cloudflare serves one at developers.cloudflare.com/llms.txt, with a product by product index where each product links to its own llms.txt.
- Next.js serves one at nextjs.org/docs/llms.txt and references it in the front matter of every docs page.
- Mintlify generates llms.txt and llms-full.txt automatically for every docs site it hosts, which covers a large share of startup documentation.
The spec page itself, updated to version 2 in August 2026, says OpenAI, Anthropic and Gemini publish llms.txt files for their developer docs. Outside docs, adoption is thinner and the files are often generated by plugins rather than written on purpose.
The best adoption data I've found is Ahrefs' study from June 2026, which checked 137,210 domains in its web analytics product. 28% of them had an llms.txt file returning a 200. Ahrefs notes that its customers skew technical and SEO aware, so treat 28% as an upper bound for the wider web.
Does anyone actually read it?
This is where you should be sober. No major AI company has said that its assistant looks for llms.txt when deciding what to cite or recommend.
Google's John Mueller said in a Reddit thread in April 2025, as reported by Search Engine Journal, that "AFAIK none of the AI services have said they're using LLMs.TXT" and compared the file to the keywords meta tag. In June 2026, on Google's Search Off the Record podcast, he went further and said LLM systems can't use the file to tell websites apart when choosing what to surface, calling it a way of saying "I have the best website ever." He did allow that it might help an agent once it has already landed on your site and needs to find a page or finish a task.
HubSpot, when it declined to build native support in June 2025, said it wasn't seeing enough evidence that ChatGPT, Claude or Perplexity use llms.txt for crawling.
The Ahrefs numbers back that up. Of the roughly 38,000 llms.txt files in its sample, 97% got zero requests in May 2026. Among the files that were fetched, named AI bots made up 19.5% of requests. GPTBot was the most frequent AI fetcher at 4.51% of requests, OAI-SearchBot led the retrieval bots at 0.74%, and Claude-Code fetched more than most of the others. Ahrefs is careful to say a fetch does not prove the bot read or acted on the file, so those are ceilings. No AI bot requested llms.txt on sites that didn't have one.
So the honest summary is: a few bots fetch it, mostly coding agents and GPTBot, and nobody has promised to use it for recommendations. It is a cheap bet, not a lever.
What llms.txt is not
- It is not a replacement for robots.txt. robots.txt tells crawlers what they may fetch. llms.txt tells an agent what is worth reading. They do different jobs and you want both. See our guide to which AI crawlers to allow in robots.txt.
- It is not a ranking signal. Google has said so plainly, and no AI search product has claimed otherwise.
- It is not a way to block training. Blocking is done in robots.txt with the training user agents, not here.
- It is not structured data. Product schema, Organization schema and your product feed do far more for AI shopping assistants than this file. Our post on what AI assistants need from your store covers that.
How to write one in 30 minutes
- Pick the 10 to 20 pages that answer "what do you sell, for whom, at what price, and how do I buy or get help." For a store that's your category pages, bestsellers, shipping, returns and contact. For a SaaS site it's the product overview, pricing, docs home, API reference, security page and changelog.
- Write the H1 as your brand name, nothing clever.
- Write a blockquote of two sentences: what you are, who it's for, where you ship or operate.
- Add one short paragraph of facts an assistant would otherwise have to guess: founded when, based where, price range, key differentiators. No adjectives.
- Group links under two to four H2s. Each link gets a colon and a note of under 15 words.
- Put long or secondary material under "Optional".
- Link to pages that render cleanly without JavaScript. If you have Markdown or plain HTML versions of docs, link those.
- Save as UTF-8, publish at the root, open it in a browser to confirm it returns a 200 with a text content type.
- Add the URL to a recurring calendar reminder and update it when prices, policies or top products change.
Example for a DTC store
# Northwind Coffee Roasters
> Specialty coffee roaster based in Portland, shipping whole bean and ground coffee across the US. Subscriptions, single origins and wholesale for cafes.
Founded 2016. Roasted to order, shipped within 48 hours. Prices from $16 to $24 per 12 oz bag. Free shipping on orders over $40.
## Shop
- [Single origin coffees](https://northwind.example/collections/single-origin): rotating lots, tasting notes and roast dates on each page
- [Subscriptions](https://northwind.example/pages/subscribe): weekly, fortnightly or monthly, cancel any time
- [Bestsellers](https://northwind.example/collections/bestsellers): our five most ordered coffees
## Buying from us
- [Shipping](https://northwind.example/policies/shipping-policy): US only, carriers, delivery times
- [Returns](https://northwind.example/policies/refund-policy): 30 day refund on unopened bags
- [Contact](https://northwind.example/pages/contact): email and hours
## Optional
- [Brewing guides](https://northwind.example/blogs/brewing): pour over, espresso, French press
- [Wholesale](https://northwind.example/pages/wholesale): pricing and minimums for cafes
Example for a SaaS docs site
# Ledgerline
> Ledgerline is invoicing and payment reconciliation software for agencies and consultancies with 5 to 200 staff. Connects to Stripe, Xero and QuickBooks.
Pricing is per seat with a free tier for up to 3 users. SOC 2 Type II. Data hosted in the US and EU.
## Product
- [Overview](https://ledgerline.example/product): what it does and who it's for
- [Pricing](https://ledgerline.example/pricing): plans, limits and what's included
- [Security](https://ledgerline.example/security): compliance, data residency, subprocessors
## Docs
- [Quickstart](https://ledgerline.example/docs/quickstart.md): first invoice in 10 minutes
- [API reference](https://ledgerline.example/docs/api.md): REST endpoints, auth, rate limits
- [Integrations](https://ledgerline.example/docs/integrations.md): Stripe, Xero, QuickBooks setup
## Optional
- [Changelog](https://ledgerline.example/changelog): dated release notes
- [Full documentation](https://ledgerline.example/llms-full.txt): every docs page in one file
How to publish it on each platform
Shopify
Shopify now serves these routes itself. According to a Shopify staff reply in the Shopify developer community on 20 May 2026, requests to /llms.txt on Online Store storefronts redirect to /agents.md, which Shopify treats as the main storefront discovery file for AI experiences. To customize it, create templates/agents.md.liquid in your theme. Note that theme templates are lost when you switch themes, so keep a copy. For product data in AI shopping channels, Shopify points merchants to Catalog Mapping instead, which we cover in Shopify agentic storefronts.
Webflow
Native. Go to Site settings, then SEO, then the LLMs.txt section, upload the file and save. Webflow's help article says the file must be UTF-8 and under 100 KB, and it's served at the root of your custom domain once published, not on the staging domain.
Framer
Native, but plan gated. In Site Settings go to Hosting, then Files, and upload. Framer's help article says static file hosting is on Pro and Enterprise plans, with 50 files on Pro. After publishing the file is served at /llms.txt.
Next.js
Drop llms.txt in the public folder and Next.js serves it at the root, as the Next.js project structure docs describe for static assets. If you want it generated from your CMS, add app/llms.txt/route.ts and return a text/plain response from a route handler. This is how most docs frameworks on Next do it.
HubSpot
HubSpot has no native support and, per its own community team in June 2025, isn't building it yet. Files you upload to File Manager are served from a /hubfs/ path, not the root, and that path can't be changed. HubSpot staff suggested the workaround: upload the file to File Manager, then create a URL redirect from /llms.txt to the /hubfs/ URL with the redirect style set to proxy, so the browser sees the root URL. That depends on having the URL redirects tool in your HubSpot plan. If you're on the free CMS and can't set a proxy redirect, you can't serve a root .txt file at all, so your options are to skip llms.txt or to put Cloudflare in front of the domain and serve the file from a Worker. Given how little evidence there is that assistants read it, skipping it on free HubSpot is a reasonable call. Spend the time on schema and page copy instead.
Where ROAS Expert fits
llms.txt is one line item in our agentic SEO work. The pieces that move citations are crawler access, structured data, clean product and pricing pages, and the right content for the questions buyers ask assistants, which we lay out for SaaS in AI visibility for SaaS and AI startups. If you want to know where you stand, an audit starts at $1,000 and includes a crawler and discoverability check.
Questions
Does llms.txt help SEO?
No. Google's John Mueller has compared it to the keywords meta tag and said it is not something search systems use. It has no effect on rankings.
Do ChatGPT, Claude or Perplexity read llms.txt?
None of them has said so. Ahrefs' June 2026 study found that GPTBot, OAI-SearchBot and Claude-Code fetched a small share of existing llms.txt files, but fetching doesn't prove the content was used.
What's the difference between llms.txt and llms-full.txt?
llms.txt is a short index of links with notes. llms-full.txt, as defined by docs hosts like Mintlify, is every page's full Markdown content in one file for tools that want all the context at once.
Does llms.txt replace robots.txt?
No. robots.txt controls what crawlers may fetch and is where you allow or block training bots. llms.txt only describes what's worth reading. Keep both.
Can I add llms.txt on Shopify?
Shopify serves the route itself and redirects it to /agents.md. You customize the content with a templates/agents.md.liquid file in your theme.
