If Your AI Search Audit Needs a Presenter It Has Failed the Handover Test

An AI search audit has not succeeded because it found your brand in ChatGPT, missed it in an AI Overview or produced 60 screenshots before lunch. It succeeds when somebody who did not attend the presentation can inspect a finding, understand its limits and make a defensible business decision from it.

The auditor begins with custody of the method. They know which questions were asked, which runs were ignored, how answers were classified and where the evidence came from. The client is then asked to fund content, technical work or reputation changes on the strength of that private understanding. Unless the report transfers enough of the method to the client, the client receives conclusions without control.

This is why a handsome AI-visibility deck can be commercially dangerous. The problem is not that the observations are necessarily wrong. The problem is that nobody can tell which recommendation would survive without the person selling it. I would rather accept a modest finding with a complete evidence trail than an exciting claim about “AI authority” that cannot be reconstructed. The modest finding can govern work. The exciting claim can only govern a sales meeting.

Visibility is not yet a business question

“How visible are we in AI search?” sounds specific because it contains a fashionable noun. It is not specific enough to commission an audit. Visibility can mean at least four different things: whether a named brand is described accurately; whether the brand appears when providers are suggested; whether a particular page is used as a supporting source; and whether exposure contributes to a visit, enquiry or other commercial action.

Those are related questions, but they do not share a metric or a remedy. A brand may be described accurately without being recommended. A page may be cited without the brand becoming a serious buying option. A provider may appear in an answer that no relevant buyer asks. A useful visit may occur without a reporting product isolating the preceding AI feature.

My first move is therefore not to inspect the dashboard. It is to rewrite the commission as a decision: we need to decide whether inaccurate public evidence, weak provider eligibility, inaccessible source material or measurement gaps deserve attention first for this audience and offer. That sentence is less glamorous. Good. It forces the audit to separate problems that attract different owners and budgets. It also stops AI search optimisation being treated as a free-floating tactic instead of one part of a wider growth strategy.

If an auditor cannot state which decision a sample is supposed to change, I would stop the handover there. More prompts will create more observations, not more meaning.

The handover gap is where confident nonsense survives

AI-search reporting has an unusual persuasion advantage: the product can vary, the interfaces are visually compelling and the underlying systems are partly opaque. That combination makes screenshots feel like privileged access. But a screenshot is only a captured output. It does not automatically preserve the question, account state, location, language, date, follow-up context, citations or alternative runs. Crop away those conditions and an observation begins to look like a universal market fact.

The product documentation itself argues for restraint. Google says AI Overviews and AI Mode may use different models and techniques, that their responses and links can vary, and that AI Overviews do not appear for every query. Google also says there are no special technical requirements or special schema needed for inclusion beyond the documented Search requirements (Google Search Central: AI features and your website). OpenAI advises users to open cited sources because search results and citations can be incomplete, outdated or incorrect (OpenAI Help: Searching the web with ChatGPT).

That matters for AI readiness. A business is not “AI ready” because somebody installed a fashionable file or produced an AI SEO score. Readiness means the organisation can expose accurate facts, trace important claims, recognise conflicting public evidence and make controlled changes when the evidence justifies them. These caveats do not make auditing pointless. They tell us what an honest audit is for: disciplined observation under changing conditions. The sales-deck version tries to overcome uncertainty with stronger language. The professional version preserves uncertainty and still makes a useful decision. That is harder work, but it is the work I would pay for.

Use the Decision Custody Chain

I use a six-link chain to decide whether a finding can leave the auditor’s hands:

Break any link and custody has not transferred. This is more demanding than a checklist of screenshots because the chain requires an auditor to show their reasoning. It also exposes recommendations that arrived before the evidence. If the action is “publish twelve comparison pages” but the finding cannot identify a legitimate buyer comparison, a factual gap and an acceptance condition, the page count is not an audit conclusion. It is inventory.

The chain works in both directions. Read forwards to understand why an action follows. Read backwards before approving money. Start with the proposed action, recover the interpretation, inspect the evidence and ask whether the original business decision is still visible. If you cannot make the return journey, hold the recommendation.

A finding needs more than a screenshot

Every consequential observation should be stored as an evidence object another competent person can inspect. The exact format can vary, but the object should retain:

• The original question and material follow-ups
• The named product or surface
• The observation date
• Relevant language, location, account or mode conditions
• The complete output or a lossless capture
• Visible citations and destination URLs
• The analyst's plain observation
• The analyst's interpretation, separately labelled
• Limitations and contradictory samples
• The recommended action and rationale

Notice what is not on that list: a percentage with no denominator, a row of green ticks or a cropped quotation presented as the model’s settled opinion.

Suppose one sampled answer names three agencies and omits yours. The observation may be perfectly real. It does not yet prove low authority, a technical defect or a need for new pages. The question may have specified a market you do not serve. The named agencies may not satisfy the conditions. Another run may differ. The commercial audience may never use that formulation. The first defensible action is usually diagnostic: confirm business fit, inspect the public evidence that demonstrates that fit and test whether the question represents a meaningful decision. Only then should content production enter the conversation.

That order matters. When production is the auditor’s main revenue line, every observation is tempted to become a content gap. Decision custody makes that incentive visible without requiring you to accuse anybody of bad faith.

Separate what happened from what somebody thinks it means

Weak reports compress four different propositions into one confident sentence. I want them separated. Observation: on a stated date, an exact question in a named product produced a particular output. Interpretation: the output represents a defined condition within that sample. Limitation: the sample does not establish frequency, cause, buyer exposure or permanence unless the method supplies that evidence. Action: a specific investigation or change is proportionate to what is currently known.

Here is the difference in practice. “The brand has poor AI authority, so it needs a GEO content hub” is a compressed sales claim. “The brand was absent from five defined provider-discovery samples; its stated qualification for three of those questions is not supported on the relevant service pages; confirm that qualification before deciding whether those pages need stronger decision evidence” is an auditable finding. The second version is not weaker. It is more useful because it identifies the next uncertainty to remove.

Do not let one score mix incompatible jobs

Composite visibility scores are attractive because senior teams want a number they can track. The risk is not aggregation itself. The risk is combining unlike observations and then pretending the movement has one meaning. A provider mention, a correct branded description, a cited URL and a search visit are not interchangeable units of success. Each answers a different question:

I would reject a master score unless the report preserves those underlying meanings. A score is acceptable when it compresses known components for a stated decision. It becomes theatre when nobody can explain what a five-point rise authorises the company to do. The same discipline applies to Search Console and third-party platforms. Google reports traffic from its AI features within the overall Web search type rather than presenting every manual prompt observation as a separate census (Google Search Central: AI features and your website). A reporting deck should distinguish direct platform exports, auditor-collected observations, estimates and calculated indices. Putting them on one chart does not make them the same kind of evidence.

Make recommendations compete for the right to exist

Most audit action lists are too long because findings are treated as discoveries to celebrate rather than claims to challenge. I would make each recommendation survive three opponents:

This changes the tone of the report. Instead of “here are 47 opportunities,” the audit becomes a ranked set of decisions with explicit dependencies. It also improves creative and web work. A website design team should not be asked to build an elaborate comparison experience when the business has not agreed which comparisons it can substantiate. Writers should not manufacture branding claims before the underlying evidence exists. Developers should not install fashionable markup when official documentation says no special markup is required. Performance marketing and customer acquisition teams should not be given an AI-visibility number that cannot be connected to audience quality or a commercial decision.

The trade-off is obvious: this process produces fewer immediately billable tasks. I accept that trade. An audit should reduce uncertainty before it increases production.

Run the three-recommendation walk-back

You do not need to reperform an entire audit to test its handover quality. Choose the three recommendations with the highest cost, reputation risk or operational consequence. For each one, walk backwards through the Decision Custody Chain using this review sheet:

Score no points for confident delivery. Score the transfer. If the presenter has to explain what the chart “really means,” the report is incomplete. If the raw evidence cannot be supplied, hold the finding. If a recommendation survives only because nobody has opened the cited source, reject it. If the action remains sensible after the strongest counter-evidence is added, it has earned consideration.

What I would approve

I would approve an AI search audit that is narrower than the original sales promise if it gives the business better control of the next decision. That is the contribution AI marketing should make to digital growth: not another isolated dashboard, but a better way to decide which public evidence, website changes and acquisition questions deserve attention. I would accept variable outputs, incomplete measurement and unresolved causes when those limits are honestly carried into the recommendation.

I would not approve invented certainty about model reasoning. I would not accept a screenshot gallery as a method. I would not fund page production merely because a brand was absent from one loosely defined answer. And I would not let two strong findings smuggle 20 speculative tasks into a delivery plan.

There is a boundary to this position. A rigorous handover cannot guarantee that every future answer will repeat the sampled result, that a corrected page will be selected or that a citation will generate revenue. It can do something more useful: make the company’s response proportionate to the evidence available today. That is the product I think an AI search audit should sell. Not privileged certainty. Transferable judgement.

Before approving the next one, close the presentation and hand the three most consequential recommendations to somebody who was not in the room. If they can reconstruct the decision without calling the auditor, custody has transferred. If they cannot, the audit is still a performance.