Site audit checklist for answer engines
llms.txt, schema, robots and metadata — technical basics for AEO.
Why technical SEO is not enough for AEO
Classic technical SEO ensures Google can crawl, index, and understand your site. Answer engines add another retrieval path: models fetch or recall pages when generating live answers in ChatGPT, Gemini, Perplexity, and Claude. If your robots.txt blocks relevant agents, or key pages lack extractable structure, you may be invisible in AI despite green Core Web Vitals.
VStok's site audit complements prompt-based visibility audits by checking AEO-oriented signals on your domain. It does not replace Search Console, but it aligns engineering work with what marketing sees in AI mention reports.
Use this checklist before major launches and after site migrations — broken schema and missing llms.txt have an outsized impact on citation-based engines.
llms.txt and machine-readable discovery
llms.txt is a convention for pointing LLM crawlers to authoritative paths: product overview, pricing, docs, comparison pages, and policies. Place it at your site root and keep it updated when URLs change. Think of it as a sitemap tuned for answer engines, not just search bots.
Pair llms.txt with a concise /llms-full.txt or equivalent deep index only if you maintain it — stale full dumps hurt more than help. Link priority pages with plain-language descriptions so retrieval systems know which document answers which buyer question.
VStok site audit verifies presence, reachability, and basic structure. Fix 404s on listed paths immediately; models cache negative signals.
Schema, metadata, and extractable content
Implement Organization, Product, SoftwareApplication, and FAQ schema where accurate. JSON-LD in the head is the most common pattern. Validate with Google's rich results test — many AI pipelines reuse the same structured graph.
Titles and meta descriptions still matter as snippets when models quote page context. Use explicit product names, category keywords, and differentiation in the first 160 characters. Avoid vague taglines that models cannot map to buyer prompts.
On-page structure: one H1, question-shaped H2s, short paragraphs, comparison tables, and visible publish or update dates. Answer engines extract blocks, not prose novels. Technical docs should expose API capabilities in plain tables, not only in PDFs behind forms.
Robots, crawling, and post-launch monitoring
Audit robots.txt for unintended blocks on /pricing, /docs, /blog, and comparison URLs. Some teams block AI training crawlers but allow search crawlers — policies vary, but know your stance and apply it consistently. Document decisions so marketing does not fight engineering by accident.
Canonical tags and hreflang prevent duplicate URLs from diluting the page models should cite. After replatforming, run site audit and a VStok visibility audit in the same week to catch regressions early.
Schedule quarterly technical reviews alongside content refreshes. AEO technical debt accumulates quietly — new subdomains without schema, PDF-only whitepapers, client-side rendering without SSR for key pages. Treat site audit failures as P1 when commercial prompts already show competitor citations replacing yours.