Obenan BriefingFlagship Briefing
A visible AI answer is not a customer
Being named in an AI answer is one event. Winning a customer is another. A US Google Search experiment measured clicks to publishers, not customers.
The one line
If a report can only tell you that you were named, then that is what you know. Everything after it needs its own number.
The evidence
18.8percentage points lower
External click-through for participants assigned to AI Mode, compared with ordinary Google Search.
SourceFull arXiv HTML paperPreprint on arXiv, submitted 18 August 2026. US adults using Chrome and Google Search, seven-day treatment in March 2026. Checked 30 August 2026.
- Published
- September 5, 2026
- Format
- Flagship Briefing
- Sources
- 2 primary, public
- Evidence basis
- Preregistered US field experiment, public preprint
01The scene
An answer can finish the job a click used to start
Hypothetical sceneAn imagined situation used to explain the mechanism. No real business, customer or answer is described.
Picture a hypothetical case: an owner opens a visibility report and sees the business named in an AI answer. The report shows one event, and it is a real one. The name appeared. It does not show that anyone left the answer, visited the site, called, booked, or arrived.
What changed is where the answer ends. When search itself supplies a complete enough answer, continuing to a website becomes optional, and a report can look busy while nothing further moves. The useful question is no longer only whether you are visible. It is where the chain from mention to customer holds, and where it stops.
02The evidence
Five states separate a mention from a customer
Between an AI answer and a paying customer there are five separate states:
Five states between an answer and a customer
- Stage 1
Named
the answer mentions the business.
Not counted
- Stage 2
Cited
the answer points to a source associated with it.
Not counted
- Stage 3
Referred
someone leaves the answer for an external site or page.
Counted
- Stage 4
Committed
someone calls, books, or orders.
Not counted
- Stage 5
Completed customer
the commitment is fulfilled and produces a real customer outcome.
Not counted
Which state the Google Search experiment actually counted. Each state can happen without the next.
- Counted
- Clicks leaving Google, measured on searches
- Not counted
- Not measured in this experiment
Each one can happen without the next. A single visibility score cannot tell you which link held and which broke, so it cannot tell you what to do about it.
03The evidence
What a Google Search experiment measured
In March 2026, researchers at the University of Pennsylvania and Northeastern University ran a field experiment with US adults who used Chrome and Google Search. For seven days, each participant was placed in one of three conditions: Google left unchanged, AI Overviews hidden, or every search redirected into AI Mode. The study was preregistered, which means the main questions and the analysis plan were published before the confirmatory results were examined.
The clearest finding is about clicks that leave Google. Compared with ordinary Google Search, being assigned to AI Mode reduced external click-through by 18.8 percentage points and reduced search sessions by 0.92 per day. Among the participants whose AI Overviews were actually hidden, the effect ran the other way: external click-through was 8.8 percentage points higher. Those two figures answer different questions and are not two halves of one clean comparison. The evidence section explains why.
The experience did not improve to compensate. People assigned to AI Mode reported lower trust, usefulness, satisfaction, sense of control, and relevance than people using Google unchanged. Hiding AI Overviews produced no detectable improvement on those measures.
What the experiment measured
- External click-through for participants assigned to AI Mode, compared with ordinary Google Search.Effect of being assigned to AI Mode, intent-to-treat
- 18.8percentage points lower
- Search sessions for participants assigned to AI Mode, compared with Google left unchanged.Effect of being assigned to AI Mode, intent-to-treat
- 0.92fewer search sessions per day
- External click-through when AI Overviews were hidden, among participants whose AI Overviews were actually hidden.Local average treatment effect; only 51.1% of AI Overviews were hidden
- 8.8percentage points higher
The two click figures are different kinds of estimate. Both are movements in percentage points, not percentage changes, and both are measured on searches. Nothing here was measured on customers.
SourceFull arXiv HTML paperPreprint on arXiv, submitted 18 August 2026. US adults using Chrome and Google Search, seven-day treatment in March 2026. Checked 30 August 2026.
04The boundary
What the experiment does not tell you
This was a US study of Chrome users, and what it counted was clicks to publishers and other websites. It did not study physical locations, local intent, Google Maps, menus, bookings, orders, purchases, or revenue, and it did not follow anyone as far as becoming a customer.
The treatment ran for seven days, so it cannot describe how habits settle over months. The sample skewed younger, more educated, and left-leaning. AI Mode carried no ads during the study, which limits comparison with other Google Search configurations. The paper is a public preprint and has not been peer reviewed, and the raw browsing data are not released.
It also does not show that correcting your business information earns a mention, a citation, a click, an order, or revenue. Accurate hours and prices are worth having on their own terms. They are not evidence of any of those outcomes.
What was counted, and what was not
Counted in the experiment
- Clicks leaving Google for publishers and other websites
- Search sessions per day
- Reported trust, usefulness, satisfaction, sense of control and relevance
Not counted
- Physical locations and local intent
- Google Maps, menus, bookings, orders, purchases and revenue
- Anyone becoming a customer
- Habits over months: the treatment ran seven days
- Ads in AI Mode: none ran during the study
05What to do
What you can usefully do
Keep the five states as five separate lanes of proof, and measure each one where it is actually recorded.
- 01For a visibility observation, keep the AI system, the interface you used, the exact question, the language, the place, the date, the answer, and the cited source.
- 02For referrals, read your own analytics instead of inferring a visit from a mention.
- 03For commitment, count calls, bookings, and orders in the systems that take them.
- 04For customers, count completed outcomes rather than intentions.
- 05Keep hours, services, prices, availability, and policies current, without treating correctness as proof that attention followed.
If a report can only tell you that you were named, then that is what you know. Everything after it needs its own number.
06In more detail
The evidence in more detail
The denominators are not interchangeable. In all, 1,444 people enrolled, completed the first survey, and installed a browser extension. Everyone spent three days on unchanged Google Search first, which gave the researchers a before picture. Of those, 1,100 made at least one search during the seven-day experiment period, and that group carries every behavioral result. Separately, 956 completed the closing survey, and that smaller group carries every reported perception. Neither number counts local-business customers.
Three groups, three kinds of result
- enrolled, completed the first survey and installed the browser extension
- 1,444
- made at least one search during the seven days. Every behavioral result rests on this group.
- 1,100
- completed the closing survey. Every reported perception rests on this group.
- 956
None of these groups counts local-business customers.
SourceFull arXiv HTML paperFull arXiv paper, methods and compliance sections. Checked 30 August 2026.
The two click figures are also different kinds of estimate. In AI Mode, 94.7% of searches were successfully redirected, so the paper reports the effect of being assigned to that condition, an intent-to-treat estimate: external click-through 18.8 percentage points lower. The condition that hid AI Overviews did not hold up. It hid about 90% of AI Overviews on the first experiment day, then Google changed the page code behind AI Overviews and the extension's hiding logic stopped working reliably; across the experiment only 51.1% of AI Overviews were hidden. The paper therefore reports the effect only where hiding actually worked, a local average treatment effect: click-through 8.8 percentage points higher. That partial failure belongs beside the number, and the two results should not be read as a symmetrical plus and minus in one A/B test.
Both figures are movements in percentage points, not percentage changes, and both are measured on searches. Nothing here was measured on customers.
07Sources
Sources
Primary, public sources only. Publication dates and the dates we checked them are kept separate so the chronology stays inspectable.
- 01arXiv abstract and submission record, submitted 18 August 2026; checked 30 August 2026.
- 02Full arXiv HTML paper, methods, results, compliance, limitations, and disclosures; checked 30 August 2026.