Key takeaways
- 1
Diagnose Google Search eligibility and ChatGPT search discovery separately; their crawlers and reports are not interchangeable.
- 2
A successful live URL test proves the page is currently accessible to Google's inspection tool, not that it has been indexed or selected for an AI answer.
- 3
When access is healthy, improve the page that answers the buyer's question with specific product facts and evidence before creating more generic content.
Diagnostic order
Find the first broken link in the chain
- 1
Discover
Internal links and a public URL
- 2
Fetch
Robots, response, and rendered content
- 3
Qualify
Index and snippet eligibility for Google
- 4
Answer
Question fit and source usefulness
First, separate the two search paths
Google says a page must be indexed and eligible to appear in Search with a snippet before it can be a supporting link in AI Overviews or AI Mode. It also says meeting those requirements does not guarantee crawling, indexing, or serving. No special AI schema or AI text file is required for those features.
OpenAI says public sites can appear in ChatGPT search and advises allowing OAI-SearchBot for summaries and snippets. Googlebot controls do not by themselves tell you whether OAI-SearchBot can fetch the site. A consumer ChatGPT search answer should be checked in the product's search experience; a model API sample is not proof of that answer.
Sources for this section: Google: AI features and your website, OpenAI: Publishers and Developers FAQ.
Turn a vague absence into a testable case
The statement 'we are missing from AI' is too broad to debug. Start with one buyer question and one page that should answer it. Write down the exact wording, the product or service the buyer is trying to evaluate, the search experience, whether a web-search mode was active, the date, region, and language. Save the answer and every visible source link. If the answer did not appear at all, record that too. This creates a case you can revisit instead of relying on memory or a one-off screenshot.
Next identify the most relevant canonical page on your site. A homepage may describe the company yet lack the specific product fact needed for a comparison question. The target could instead be a product page, pricing explanation, integration guide, or clearly sourced article. Ask a colleague to read that page without internal context and answer the buyer's question. If they cannot, a crawler is unlikely to invent the missing detail for you. The work may be a page rewrite, a better internal link, or an entirely different target page.
Make a small case sheet with four fields: question, expected page, observed answer, and first failing check. Keep distinct rows for Google AI Overviews and ChatGPT search. A successful observation in one system tells you little about the other. This also prevents a content team from rewriting a page when the immediate failure is a blocked URL or a wrong canonical.
Sources for this section: Google: AI features and your website, OpenAI: Publishers and Developers FAQ.
Check the URL and the rendered page
Inspect the exact canonical URL, not just the homepage. Verify it returns a successful response, renders the main answer as visible text, has a self-consistent canonical, and is linked from a relevant page. Check robots rules and any noindex or snippet preview controls. A sitemap helps discovery but does not force indexing.
For Google, use Search Console URL Inspection: compare the indexed status with Test live URL, then inspect the rendered page and selected canonical when available. Google's live test only checks current access; it cannot predict indexing or canonical selection. If a page is discovered but not indexed, improve its in-content links and usefulness, then request indexing after the live test passes.
- 01
Fetch and render
Check status, canonical, robots, noindex, and whether the answer exists in rendered HTML.
- 02
Inspect Google status
Compare indexed data and live test; do not call a live test an indexed result.
- 03
Inspect OpenAI access
Check OAI-SearchBot rules and any hosting or security layer that might block it.

Sources for this section: Google: URL Inspection tool, Google: AI features and your website, OpenAI: Publishers and Developers FAQ.
Diagnose the Google path in Search Console
Inspect the exact canonical URL in the correct Search Console property. The indexed result describes what Google knows from its last crawl; the live test checks the page as it can be fetched now. Record both outcomes and their dates. If Google selected a different canonical, inspect that URL too before assuming your preferred page is the one eligible to serve. If the live test succeeds but the indexed status remains negative, the page is accessible now but has not necessarily been processed or selected for indexing.
Open the live rendered view and check the main answer, not just the HTTP status. Look for a page that returns a success response but renders an empty shell, a soft error, or content hidden behind interaction. Check robots.txt, CDN or security rules, redirects, noindex directives, and snippet controls. For a Google AI Overview supporting link, Google's documented technical gate is indexing plus eligibility to appear in Search with a snippet. Passing that gate still does not ensure the feature appears for the query or that your page is chosen.
After a real fix, test the live URL again and request indexing where appropriate. Save the date, the rendered evidence, and the change made. Revisit the indexed result after Google has had time to crawl and process it; repeating live tests on the same day does not turn a live result into an index confirmation. If the URL is indexed, shift attention to the buyer question and the usefulness of the page rather than treating indexing as the final goal.
Sources for this section: Google: URL Inspection tool, Google: AI features and your website.
Diagnose ChatGPT search separately
OpenAI's publisher guidance names OAI-SearchBot for discovery of content used in ChatGPT search summaries and snippets. Review the robots rules for that user agent and check whether your host or security service denies it even when an ordinary browser can load the page. A Googlebot test cannot establish OpenAI crawler access. A robots rule alone also cannot tell you whether a specific answer will use the page, so keep access and answer observations as separate checks.
Test the question in the consumer ChatGPT search experience if it is available to you, and save the exact answer and linked pages. Search mode matters: a model response without web search is not evidence of current search citations. Examine whether the answer cites another page that directly states the required fact. If so, improve your own page with specific, verifiable information for the same buyer task. Do not copy the competitor's wording or create several near-duplicate pages hoping one will be selected.
Repeat the same question under recorded conditions after your page change. A single missing answer should trigger investigation, not a guaranteed-citation claim. If the answer varies across attempts, report that variability instead of choosing only the favorable screenshot. Keep the source URL and collection date with every example you show to a client or publish on your site.
Sources for this section: OpenAI: Publishers and Developers FAQ.
If access is sound, test the answer itself
Choose buyer questions that your product should answer, including problem, comparison, and vendor-selection questions. Save the exact prompt, answer, linked sources, date, region, language, and search mode. A single absence is only a sample: answers and links can vary, and some Google searches never show an AI Overview at all.
Compare the cited pages with yours. Does your page state the product category, use case, constraints, pricing or process where relevant, and proof that can be published? If competitors offer clearer facts or more direct answers, update the existing relevant page. Link it from related service, product, and guide content so readers and crawlers can find it.
Sources for this section: Google: AI features and your website.
Choose the next fix from the evidence
A blocked or erroring page needs a technical fix. A crawled page without a useful answer needs content work. An eligible, useful page that is absent from one answer needs more observation before a major rewrite. Document the change date and repeat the same question set; there is no sound way to promise a particular citation or appearance.
What you see
Fetch blocked or error
Likely work
Fix robots, security, or response
How to verify
Repeat live fetch and render
What you see
Google page not indexed
Likely work
Improve page and discovery links
How to verify
Reinspect indexed status later
What you see
Page eligible but absent
Likely work
Compare answer usefulness and sources
How to verify
Repeat the fixed question sample
Use a short recheck log before changing strategy
For each fix, record the owner, the page changed, the original failure, the evidence that the change is live, and a date to recheck. A practical sequence is: verify a successful public fetch and visible answer, confirm the relevant crawler is allowed, wait for the indexed state where Google is involved, and then repeat the fixed question set in the actual answer experience. If the technical state improves but citations do not, you still learned that access was only one part of the problem.
Avoid replacing a specific guide with a generic 'AI optimized' article unless the new page answers a distinct task. The strongest next step is usually the smallest change justified by evidence: repair a blocked page, clarify an answer, publish a missing product constraint, or improve a link from a relevant page. Preserve the before-and-after versions so the team can see what actually changed.
- 01
Confirm access
Save response, render, robots, and indexed-versus-live status for the exact URL.
- 02
Confirm usefulness
Check whether the page answers the recorded buyer question with verifiable facts.
- 03
Repeat the observation
Use the same question and recorded conditions, then describe the result without a placement guarantee.
Common questions
Does indexing guarantee an AI Overview citation?
No. Google describes indexing and snippet eligibility as requirements for a supporting link, but not a guarantee of appearing in AI Overviews or AI Mode.
Will an llms.txt file make ChatGPT cite my site?
There is no guaranteed citation mechanism. OpenAI's public guidance focuses on access for OAI-SearchBot; Google says special AI text files are not required for its AI Search features.
Why can Google index a page while ChatGPT search still misses it?
Google indexing and ChatGPT search are separate paths. Check whether the page is accessible to OAI-SearchBot, then test the exact buyer question in ChatGPT search. Even an accessible, useful page is not guaranteed to be selected or cited for every answer.
Primary sources
These product and search-platform documents were checked on 24 September 2026. Interfaces and policies can change.
- Google: AI features and your website
Eligibility and SEO best practices for AI Overviews and AI Mode.
- Google: URL Inspection tool
Indexed status, live tests, render, and request indexing.
- OpenAI: Publishers and Developers FAQ
ChatGPT search discovery and OAI-SearchBot.
Want help applying this to your site?
If the page is accessible but the important buyer questions still miss your brand, map the problem before rewriting everything.
See AI Visibility RecoveryRead next
Technical guide
Google AI Overviews eligibility: crawling, indexing, and snippetsAudit checklist
An AEO audit checklist for B2B SaaS websites

