<blue>Does llms.txt Matter?</blue> A Plain Test of the Claims

For most business websites, an llms.txt file will not change whether AI assistants find or cite you. Google says its search features ignore the file, and the largest public study found no measurable link between having one and getting AI citations. The file does have a real use: helping coding agents and developer tools read technical documentation. If you run docs or an API, it is worth a few hours. If you run a store or a services site, your time is better spent elsewhere.

Key takeaways

• llms.txt is a proposed standard, not an adopted one. It was proposed in September 2024 by Jeremy Howard of Answer.AI.
• Google's AI optimization guide lists llms.txt among things you can ignore for Google Search, including its AI features.
• An SE Ranking analysis of about 300,000 domains found no significant correlation between llms.txt and AI citations.
• The clearest benefit is for software documentation read by coding tools and agents.

What llms.txt is

llms.txt is a Markdown file placed at the root of a website, such as /llms.txt. The proposal at llmstxt.org describes it as a way "to provide information to help agents use a website." The only required element is an H1 with the site or project name. It usually adds a short summary in a blockquote, then H2 sections that list links to key pages with brief notes. A section titled "Optional" marks links an agent can skip when it needs less context.

The idea is reasonable. Websites are built for browsers, with navigation, scripts, and layout that add noise for a language model. A short Markdown index promises a cleaner path to the important pages.

Testing the claims, one by one

Claim: "Google uses llms.txt for AI Overviews and AI Mode"

Verdict: false. Google published an AI optimization guide on May 15, 2026, and added a note on llms.txt on June 15, 2026. The guide says: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them." It adds that keeping such files "will neither harm nor help your site's visibility or rankings in Google Search."

Claim: "llms.txt gets you cited more by AI assistants"

Verdict: not supported. SE Ranking analyzed roughly 300,000 domains, as reported by Search Engine Journal in November 2025. About 10.13% had an llms.txt file. Statistical tests found no significant correlation with AI citations, and removing the llms.txt variable from SE Ranking's model actually improved its accuracy. The study's own conclusion was that the file "doesn't seem to directly impact AI citation frequency. At least not yet."

Claim: "AI crawlers look for llms.txt first"

Verdict: weak evidence. Search Engine Journal's report notes that some server logs show GPTBot occasionally fetching llms.txt files, but not often and with no visible tie to citations. As of November 2026, we could not find a statement from OpenAI, Anthropic, or Perplexity saying their answer products rely on llms.txt. The AI labs do publish llms.txt files for their own developer documentation, which shows support for the format in that setting, not that their assistants read it on your site.

Claim: "Lighthouse now requires llms.txt"

Verdict: overstated. Chrome's Lighthouse added an llms.txt check in its agentic browsing audits. According to Chrome's documentation, it flags a page only if a server error occurs when fetching the file. A missing file returns "Not Applicable," because "providing the file is optional at the moment."

Claim: "It helps coding agents and developer tools"

Verdict: plausible, and the strongest use case. The llmstxt.org proposal itself says the files "are used most heavily for software documentation." Developer platforms publish them so coding assistants can pull the right reference pages into context. If your product has an API, SDK, or technical docs, that audience is real.

When an llms.txt file is worth creating

• Worth doing: API references, developer docs, SaaS help centers, and products where customers use AI coding tools to integrate with you.
• Optional: content-heavy sites that already maintain a clean sitemap and want an extra index. It costs little and, per Google, does no harm.
• Low priority: ecommerce stores, local services, and hospitality sites. AI assistants discover these through search indexes, product feeds, and third-party sources, not llms.txt.

Hypothetical example: A B2B software company has 40 hours to spend on AI visibility this quarter. Writing llms.txt for its API docs takes about four. The other 36 go to fixing pages that render content only with JavaScript, adding Organization and Product schema, rewriting top support articles in an answer-first format, and tracking 30 buyer prompts across ChatGPT, Gemini, and Perplexity. The llms.txt file serves developers. The rest is what moves citations.

What moves AI visibility instead

• Crawler access. Make sure search bots such as OAI-SearchBot, Claude-SearchBot, and PerplexityBot are allowed in robots.txt and not blocked by your CDN.
• Readable HTML. Key content should be present in the page source, not loaded only by scripts.
• Answer-first content. Clear headings, direct answers, and self-contained passages are easier to extract and quote.
• Entity consistency. The same name, description, and facts across your site, schema, and third-party profiles.
• Measurement. Prompt-level tracking shows whether any change worked.

The AEO and GEO audit checklist covers each of these in a 26-point format, and the guide to SEO strategy with engineered analytics explains how to measure traffic from LLMs once the work is done.

How to run your own test

1. Publish llms.txt and note the date.
2. Watch server logs for requests to /llms.txt and record which user agents ask for it.
3. Track a fixed set of buyer prompts across assistants for 60 to 90 days, before and after.
4. Compare citation rates with pages and changes you did not touch, so seasonality does not fool you.

If the logs show no requests and citations do not move, the file is doing nothing for discovery on your site, whatever a vendor promises.

How Onward approaches it

Onward's answer engine optimization services start with tracked prompts and measured visibility, then prioritize work by what changes citations and customers. That approach produced the results in the Villa Alberti AEO case study, where AI visibility rose 15x and AI answers drove more than $100K in bookings.

Frequently asked questions

Can an llms.txt file hurt my site?

Google says it will neither harm nor help visibility in Google Search. The main cost is maintenance: an outdated file that points to moved or deleted pages is worse than none for the tools that do read it.

What is llms-full.txt?

It is a related convention where a site publishes the full text of its documentation in one Markdown file. It serves the same developer use case and has the same lack of evidence for general AI search visibility.

Does Shopify or WordPress create llms.txt automatically?

Some plugins and platforms can generate one. Check what it contains before relying on it, and remember that generating the file does not change how assistants discover your products.

The bottom line

llms.txt is a sensible idea with little evidence behind its biggest promises. Publish one if developers and coding agents use your documentation. Otherwise, spend the time on crawler access, readable pages, consistent entity data, and prompt tracking.

If you want to know what is actually limiting your AI visibility, schedule a free audit with Onward.