llms.txt implementation guide for AI discoverability
How to publish llms.txt and llms-full.txt so answer engines and AI crawlers can find authoritative product information.
What llms.txt is and why it matters
llms.txt is a plain-text index at your site root — similar in spirit to robots.txt — that points AI systems and crawlers to your most important pages, product summary and contact information. It helps models resolve factual claims about your brand when retrieval is enabled.
A companion file, llms-full.txt, can include extended documentation: pricing, API overview, FAQs and deep links to guides. Together they practice what AEO teams preach: make authoritative information machine-readable.
VStok publishes both files at vstok.net/llms.txt and vstok.net/llms-full.txt and audits customer sites for llms.txt presence in Site Audit.
Recommended structure
Start with a one-line product summary in blockquote style, then group links under headings: Product, Platform features, Resources, Legal and Contact. Use markdown links with descriptive labels — not bare URLs.
Include your pricing page, signup flow, top guides and API docs. Add a Last-Updated date so crawlers know freshness. Mirror critical FAQs from your homepage in llms-full.txt for models that ingest the extended file.
Keep llms.txt concise — under 150 lines — and push depth to llms-full.txt. Link between the two files explicitly.
Technical deployment
Serve both files as text/plain with UTF-8 charset. Static generation or edge caching is fine; avoid auth gates. Confirm 200 responses in production and list the URLs in your sitemap if you maintain a dynamic sitemap.
Pair llms.txt with schema markup, accurate metadata and robots rules that allow reputable AI crawlers. llms.txt does not replace SEO — it complements structured data and strong owned content.
Submit updated URLs via IndexNow or your search console after major product launches so Bing and partners recrawl quickly.
Measuring impact
Track whether citation sources in VStok audits begin referencing your domain after publishing llms.txt. Monitor AI visibility on prompts about your product category and branded queries.
Revisit files quarterly when pricing, features or legal entities change. Stale llms.txt is worse than none — it can reinforce outdated facts.
Use VStok's free LLM SEO audit to preview whether your site exposes llms.txt and related AEO signals before full monitoring.