What is llms.txt? A practical guide to publishing one—and honest limits for AI citations and AEO

When people ask “what is llms.txt?”, they usually want two things at once: a format definition and a growth promise. The format is real and useful. The growth promise is where teams get hurt. llms.txt is a markdown file that offers a concise, expert-oriented entry point so agents can find the right detail pages without scraping navigation chrome. It is closer in spirit to a curated README for your website than to a secret handshake that inserts your brand into ChatGPT.

The public proposal at llmstxt.org (v2) describes why the file exists: HTML pages wrap information in navigation, ads, and scripts; context windows are finite; agents benefit from concise guidance plus links to clean markdown. The same proposal is explicit that llms.txt coexists with robots.txt and sitemaps—it does not replace crawl permissions or a full URL inventory.

For Answer Engine Optimization, treat llms.txt as one discovery aid among many: clear entities, crawlable helpful pages, consistent facts, and corroboration still do the heavy lifting. Google’s AI features documentation says you do not need new machine-readable AI files to appear in AI Overviews or AI Mode. Google’s generative AI optimization guide goes further for Search: Google Search ignores llms.txt; maintaining one for other systems is fine and will neither help nor harm Google Search visibility. That is the honest baseline this article will not contradict.

According to llmstxt.org, an llms.txt file uses Markdown with a specific section order so both language models and classical parsers can read it. At minimum you need an H1 with the project or site name. Strong files then include a blockquote summary with key facts, optional notes without extra headings, and zero or more H2 sections that contain markdown link lists. Each list item is a required hyperlink, optionally followed by a colon and notes.

Location matters. The proposal allows /llms.txt at the site root or an llms.txt under a path (for example /docs/llms.txt) covering URLs under that path. Where more than one file applies, agents should prefer the most specific. That path design is intentional: authors on shared hosts who cannot write /.well-known/ can still publish an llms.txt beside the content they control.

The proposal further encourages clean markdown versions of important pages—either by appending .md (page.html.md) or replacing the extension (page.md)—and suggests link relations such as rel="alternate" type="text/markdown" for page mirrors and rel="describedby" pointing at the governing llms.txt. Those conventions help clients discover markdown without guessing. They are still conventions in a community proposal, not a Google Search ranking requirement.

Start with the facts you already want consistent everywhere: legal name, product category, what you are not (same-name disambiguation), canonical site URL, and the handful of pages that define you. If those facts are wrong on /about, schema, and review profiles, an elegant llms.txt will mainly help agents retrieve the wrong story faster. Fix substance first; then curate.

Draft the H1 and blockquote so a cold agent can state what you do in one breath. Prefer concrete nouns over slogan stacks. If you operate in a collision-prone name space, put the exclusion in the summary—TopFound’s own file states it is the AEO agency at topfound.io and not unrelated same-name organizations.

Build H2 link lists that match how agents work: official information, services or docs, policies, machine-readable discovery. Link to markdown-friendly targets when you have them. Keep the root llms.txt small enough to fit comfortably in context; put the long corpus behind a link such as llms-full.txt or individual .md mirrors.

Ship the file at a stable URL, ensure it returns text/markdown or plain text without an interstitial, and verify it is allowed in robots.txt for the agents you care about. Re-fetch after deploys. Add internal reminders to update llms.txt when pricing, positioning, or canonical URLs change—stale discovery files create confident wrong answers.

Test with a realistic agent workflow: give only your llms.txt as a starting point and ask questions a buyer would ask. Note where the agent follows the wrong link or invents a fact you never linked. That test is recommended in the public proposal and is more honest than assuming publication alone changes model behavior.

It can reduce wasted tokens. Agents that fetch a curated map spend less time reconstructing your information architecture from nav bars and cookie banners. For documentation-heavy products, that is often the original use case the proposal emphasizes.

It can encode disambiguation. A short, explicit “who we are / who we are not” block helps when your brand name collides with other entities—an AEO problem TopFound also teaches on /blog/what-is-topfound-entity-disambiguation. The file does not magically merge knowledge graphs, but it gives retrieving systems a clean first-party statement.

It can point agents at citation-worthy pages: pricing with dates, policy pages, case-study hubs, and definition URLs. That is useful when your best answers are buried three clicks deep. Pair the links with pages that are actually helpful and people-first; the map cannot redeem thin targets.

It can support your broader machine-readable posture—markdown mirrors, well-known catalogs, API catalogs—without pretending those files are Google ranking factors. TopFound publishes /llms.txt as a curated index and /llms-full.txt as a fuller markdown corpus so agents can choose brevity or depth.

It cannot force ChatGPT, Perplexity, Gemini, Claude, Grok, Copilot, or Google AI Overviews to cite you. Answer engines select sources with their own retrieval and ranking systems. A discovery file is an input some clients may fetch; it is not a placement API.

It cannot replace SEO fundamentals. If pages are blocked, unindexable, or unhelpful, you remain ineligible for Search features that depend on normal indexing and snippets. Google’s AI features guide states there are no additional technical requirements beyond ordinary Search eligibility for AI Overviews and AI Mode—and that meeting requirements still does not guarantee serving.

It cannot override Google Search’s documented non-use. The generative AI optimization guide’s mythbusting section lists llms.txt among things you do not need for Google Search, and states Google Search ignores these files. If your only KPI is Google AI Overviews impressions, llms.txt is the wrong lever to celebrate. If you also care about coding agents, doc assistants, and other systems that voluntarily fetch llms.txt, the file can still be worth maintaining for those audiences.

It cannot fix contradictory facts. If /llms.txt says Growth starts at $797/month and a stale landing page says something else, you have built a higher-confidence contradiction. AEO programs should treat the file as part of the same editorial governance as pricing and /llm-info.

It cannot substitute for measurement. Track whether agents retrieve the file, whether answers improve on a frozen prompt set, and whether referrals move—as separate rows. Publication day is not a results day.

TopFound publishes a live /llms.txt that opens with the agency name, a blockquote summary of managed AEO/GEO/AI SEO positioning, and curated sections for official information, services, facts, how it works, audiences, results exhibits, FAQ, contact, policy, and machine-readable discovery. The file explicitly points agents to /llms-full.txt for fuller page-level markdown and to /llm-info as an authoritative facts page for assistants.

That split matches the proposal’s intent: keep the root map small; put depth behind links. TopFound also serves markdown mirrors for pages (for example paths with a .md suffix) so an agent following a link can retrieve clean text. None of these files are presented publicly as a ChatGPT ranking hack. They are discovery and clarity aids inside a program that still ships answer-ready pages, technical SEO, entity cleanup, and off-site corroboration.

If you use TopFound’s file as a reference pattern, copy the discipline—not the claims. Put your own pricing, guarantees, and proof behind URLs you maintain. Keep disambiguation lines updated. When blog posts or resources launch, add them to the curated lists so agents do not rely on an outdated map while your HTML hub is already current.

Label: fictional developer-tools company. PortalDocs launched /llms.txt in a week, modeled on public examples, and announced “AI SEO complete” internally. An agent test using only the file answered pricing correctly but invented an enterprise SKU that existed only in an old PDF linked from the Optional section. ChatGPT citation rate on their frozen buyer prompts did not move in the following month—which should not have surprised anyone who read Google’s non-use note for Search or who understood that consumer chat products do not pledge to honor every site’s discovery file.

The useful second sprint was editorial. They removed stale PDF links, aligned the blockquote with the live pricing page, added a “Who we are not” line for a same-name community project, and linked markdown mirrors for the three tutorials that already earned newsletter traffic. They kept llms.txt, but they stopped treating publication as the KPI. The KPI became: agent tests with fewer invented facts, plus separate tracking of mentions/citations/referrals on the frozen prompt set.

Copy the lesson: the file amplifies whatever truthfulness you already practice. If your program needs citations, invest in citable pages and corroboration; use llms.txt to help agents find them—not to skip them.

Sequence matters. Fix crawl access, indexation, and helpful content first. Clarify entities and on-page answers for buying prompts. Add corroboration where competitors win because third parties explain you better than you explain yourself. Publish and maintain llms.txt as part of machine-readable clarity—alongside schema that matches visible text, not instead of it.

Governance matters. Assign an owner. Add llms.txt to the same release checklist as pricing and policy updates. When you launch educational URLs, decide whether they belong in the curated root map or only in the full corpus. A map that lists everything becomes a sitemap with worse tooling.

Measurement matters. Log whether important agents fetch the file (where logs allow), whether answer accuracy improves on identity prompts, and whether commercial prompt mentions/citations move after substance changes. Keep those rows separate—see /blog/ai-search-analytics for the category buyer guide and /resources/how-to-measure-ai-search-visibility for methods. Do not attribute a Google AI Overview appearance to llms.txt when Google states it ignores the file for Search.

llms.txt is a practical, proposal-driven markdown map that helps agents find concise, curated truths about your site. Learn the format, publish it at a stable path, link to clean detail, and maintain it like any other source-of-truth surface. Pair it with /llms-full.txt or page-level markdown if you need depth without bloating the root file—the pattern TopFound uses in public.

Do not confuse that good hygiene with a citation guarantee. Google Search documents that it ignores llms.txt for Search and generative AI features in Search; other systems may still benefit from the file on demand. AEO still rests on helpful pages, clear entities, technical eligibility, and corroboration you can retest. Publish the map. Earn the answer. Measure each row without mythology.