You can't measure AI search with one number: five instruments and what each one can't see
AI visibility estimates need roughly 40 to 150 queries per engine, with 95% confidence intervals of 3 to 6 points, so most weekly movement is noise. Five instruments measure AI search, including the search platforms' own reports, and each misses what another catches. Only incrementality testing shows cause.
No single instrument measures AI search, and each one is noisier than most reports admit. Reliable citation-share estimates need roughly 40 to 150 queries per engine, with typical 95% confidence intervals of 3 to 6 percentage points Sielinski · IQRush · 2026. Five instruments now measure AI search: prompt tracking, server logfiles, web analytics, self-reported attribution and the AI reports Google and Bing publish themselves. Each has a blind spot, and only incrementality testing supports a causal claim.
Prompt tracking is bounded by the prompt set and by how each prompt is run
Prompt tracking measures how often a brand appears in answers to prompts the customer chose, not prompts buyers typed. Otterly's help documentation states there is "no way to learn which prompts are most asked at ChatGPT or Perplexity" Otterly AI · 2024. It is still the only instrument that produces a competitor benchmark.
How a prompt is run changes the result. Logged-out ChatGPT showed evidence of search in about 58% of answers to 100 entity-comparison prompts Druck · Graphite · 2026. On prompts that never search naturally, forcing search shifted visibility by 20 points on average Druck · Graphite · 2026. That is one vendor's test of one engine, with a disclosed design.
The surface matters as well. Across 52,170 chats, the average brand's visibility differed by 41% between ChatGPT's interface and its API Landwehr · Peec AI · 2026. The study discloses no temperature, search setting or model variant, and its author's company sells interface-based tracking. Treat it as provisional.
Logfiles and Google Analytics miss most of what AI search does
Server logs are unsampled records of which AI crawlers fetched which pages, and they stop there. Anthropic's ClaudeBot crawled around 70,000 pages for every visitor it referred, and OpenAI's GPTBot around 1,700 Cloudflare · 2025. A log entry cannot show whether the page shaped the answer or how the brand was described.
Google Analytics now has an AI Assistant channel, assigned only when a visit's referrer matches Google's list of assistants Google · 2026. Visits with no referrer still land in Direct. Google files its own AI Overviews and AI Mode under Organic Search, so the channel is a floor, not a total.
The size of the hidden share is contested. Loamly reports that 70.6% of 20,428 AI visits arrived without a referrer Di Cesare · Loamly · 2025. Loamly's own classifier assigned those visits to AI from timing and behaviour signals, so the figure is an inference, not a count Di Cesare · Loamly · 2026.
Tagging can also fail overnight. From 9 October 2026, GA4 ChatGPT sessions fell 22% to 57% on sites Graphite tracked Southern · Search Engine Journal · 2026. Adobe data on about 500 pages showed ChatGPT visits down about 31,000 a day while Direct rose about 26,000 Robinson · Evertune · 2026. ChatGPT's tracking tags reappeared on 16 October, and most of the incident's figures come from tool vendors.
Self-reported attribution sees the buying decision, and it runs on memory
Asking buyers where they heard about a brand is the only method that captures an assistant conversation followed by a direct visit. Clicks are scarce by comparison: across 81,947 websites, AI traffic averaged 0.25% of total traffic Ahrefs · 2025. The influence sits upstream of the click.
The behaviour self-report captures is common. Among 1,076 B2B software buyers, 51% now start research with an AI chatbot more often than with Google Sanders · G2 · 2026. On the assistant's guidance, 69% chose a different vendor than planned. G2 sells answer-engine products, so the survey is directional.
The weakness is the respondent. Self-report captures what a buyer remembers and will write down, which favours the most recent and most nameable touch. It carries the recall limits of every stated-preference measure, and it cannot be benchmarked against a competitor.
Google and Bing AI performance reports count impressions and citations, not clicks
Search Console's generative AI report counts impressions for pages shown in AI Overviews, AI Mode and generative features in Discover Google · 2026. It segments them by page, country, device and date, and it shows no clicks, click-through rate, position or queries. Google states these impressions were already inside the existing Search Console totals. The report launched in June 2026 and reached all sites worldwide on 31 August.
Bing Webmaster Tools has counted citations since February 2026, across Copilot, Bing's AI summaries and partner integrations Microsoft · 2026. It adds page-level citation counts and grounding queries, the phrases the AI used to retrieve cited content. Microsoft describes the grounding queries as a sample of overall citation activity.
Google counts a link being shown, while Bing counts a citation displayed, so the two numbers do not add up. Neither report covers ChatGPT, Perplexity or Claude, and neither shows a competitor. They are the only first-party record of AI visibility, and each describes one company's surfaces.
An AI visibility percentage without a sample size is not a measurement
Citation share behaves as a sample estimate. Reliable estimates needed roughly 40 to 50 queries on Gemini, about 100 on Perplexity and 150 or more on SearchGPT Sielinski · IQRush · 2026. The same work found apparent week-over-week changes often fell inside sampling noise. Single snapshots understate brand presence, and run-to-run variation is a parameter to measure Schulte et al. · University of St. Gallen · 2026.
Running the same prompt every day buys little. Across 12,933 responses, a repeat past the fifth changed the estimate by 0.0003 Żatuchin · Estonian Entrepreneurship University of Applied Sciences (EUAS); Rankfor.AI · 2026. Adding languages cut error variance about fifteen times as much as five more repeats. That study measured sentiment for 20 Central and Eastern European brands, so its language finding partly reflects its design. Work on agentic evaluations recommends 5 to 10 repeated runs Mustahsan · arXiv · 2025.
Only incrementality testing supports a causal claim about AI search work
The five instruments describe correlation, and none isolates the effect of the work that was done. Search Engine Land's five-layer framework of May 2026 ends with portfolio-level incrementality testing for that reason DeMott · Search Engine Land · 2026. It compares properties with different levels of investment, and few single brands have the comparable properties it needs.
One log-based study shows why the control matters. On a single domain, ChatGPT referrals grew 5.7 times while untreated pages on the same domain grew 3.5 times Watanabe · 2026. The estimated extra effect of the optimisation work was 1.82 times, and a placebo test was not significant. It covers one domain and is not peer reviewed, but most of the raw growth was platform tailwind.
FAQFrequently Asked Questions
Sources
Sources are tiered per our methodology & sources page.
Where Does the Noise Come From? A Variance-Components Decomposition of Non-Determinism in LLM Brand Answers (arXiv 2607.13304)
Estonian Entrepreneurship University of Applied Sciences (EUAS); Rankfor.AI · Dmitrij Żatuchin · 2026
Across 12,933 responses covering 20 brands, 8 languages and 3 engines at temperature 0.3, within-prompt resampling explained 34.8% of outcome variance and query language 31.6%, while brand identity explained 0.7%. Repeats past the fifth added almost nothing; adding languages cut relative-error variance roughly fifteen times more than five extra repeats.
Methodology note
Preprint by Dmitrij Żatuchin (Estonian Entrepreneurship University of Applied Sciences and Rankfor.AI), submitted 2026-07-14. A crossed random-effects model partitioned variance in brand sentiment across resampling, paraphrase, model identity and query language, using GPT-5.2, Gemini 3 Flash and Perplexity. Fetched and read directly.
Quantifying Uncertainty in AI Visibility: A Statistical Framework for Generative Search Measurement (arXiv 2603.08924)
IQRush · Ronald Sielinski · 2026
Citation visibility behaves as a sample estimate, not a fixed value. Reliable citation-share estimates required roughly 40 to 50 queries on Gemini, about 100 on Perplexity, and 150 or more on SearchGPT. Typical 95% confidence intervals spanned 3 to 6 percentage points, and apparent week-over-week movements often fell inside sampling noise.
Methodology note
Preprint by Ronald Sielinski (IQRush), submitted 2026-06-09. Bootstrap resampling built 95% confidence intervals around citation share and prevalence across Perplexity Search, OpenAI SearchGPT and Google Gemini, using 200 queries per topic across three consumer topics, sampled daily over nine days plus ten-minute-interval runs. Fetched and read directly.
Introducing Search Generative AI Performance Reports in Search Console
Google · 2026
Google Search Console now reports impressions for URLs served as sources in AI Overviews, AI Mode and generative features in Discover, segmented by page, country, device and date. The launch excludes clicks, CTR, per-link position and all query-level data, and Google confirms these impressions were already inside existing Search Console totals.
Methodology note
Official Google Search Central announcement, 3 June 2026, rolled out first to a subset of UK site owners under Competition and Markets Authority pressure. Direct fetch returned only the archive index, so scope, metrics and limitations were confirmed against two independent secondary reports naming the same launch date and constraints.
Google Analytics' default channel group now includes an AI Assistant channel for visits from sources like ChatGPT, Gemini, Deepseek, Copilot or Grok, assigned when the referrer matches Google's list. It explicitly excludes Google's AI Overviews and AI Mode, which Google classifies as Organic Search. Sessions with no referrer still fall into Direct, so the channel undercounts AI-originated visits.
Methodology note
Official Google Analytics Help Center page defining GA4 default channel rules, read directly on 2026-10-12. The AI Assistant definition sets medium to ai-assistant and campaign to (ai-assistant) when the referrer matches an unpublished list of AI assistants. Claude and Perplexity are not among the named examples. The page is undated; launch timing comes from trade-press coverage.
Don't Measure Once: Measuring Visibility in AI Search (GEO)
University of St. Gallen · Schulte et al. · 2026
Argues that single-snapshot AI visibility measurement understates true brand presence in generative search. Proposes a longitudinal measurement framework that captures variation across runs, prompts, and platforms, demonstrating that any one-time snapshot of citation rate or mention rate can swing materially across repeated queries. Stochasticity itself is a measurement parameter, not noise to discard.
Methodology note
arXiv preprint 2604.07585 (April 2026). Position paper proposing a multi-run, multi-prompt evaluation protocol for GEO. Direct fetch on arxiv.org returned the canonical abstract page; PDF body was inaccessible but methodology summary was confirmed through the abstract and the linked DOI.
Introducing AI Performance in Bing Webmaster Tools (Public Preview)
Microsoft · 2026
Bing Webmaster Tools added an AI Performance dashboard in public preview on February 10, 2026. It shows total citations of a publisher's site across Microsoft Copilot, AI-generated summaries in Bing, and partner integrations, plus average cited pages per day, page-level citation counts, and grounding query phrases. Publishers can use the data to see which pages are referenced in AI answers and how that activity changes over time.
Methodology note
Official Microsoft Bing Webmaster blog post by Krishna Madhavan, Meenaz Merchant, Fabrice Canel, and Saral Nigam, announcing the public preview. The post positions AI Performance as an early Generative Engine Optimization tool, and references IndexNow as the recommended way to keep cited content fresh.
Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation
arXiv · Mustahsan · 2025
Quantifies stochasticity in agentic LLM evaluations using intraclass correlation coefficients (ICC). Shows that single-run evaluations of agentic systems are unreliable because run-to-run variance is large relative to the gap between system variants. Recommends a minimum of 5 to 10 repeated runs per evaluation and reports the ICCs for several common agentic benchmarks.
Methodology note
arXiv preprint 2512.06710 (December 2025). Direct fetch on arxiv.org returned the abstract page. The paper applies the intraclass correlation coefficient framework from psychometrics to LLM agent evaluation and reports ICC values across multiple published benchmarks.
The Crawl-to-Click Gap: Cloudflare Data on AI Bots, Training, and Referrals
Cloudflare · 2025
AI crawlers read content far more than they send referrals back. Anthropic's ClaudeBot crawled around 70,000 pages for every visitor it referred; OpenAI's GPTBot crawled around 1,700 for every visitor; Perplexity around 5 for every visitor. Mistral was the only major AI engine where referrals outweighed crawl volume.
Methodology note
Aggregate analysis of crawl requests and referral traffic across the Cloudflare network. For each major AI crawler, the team divided pages crawled by visits sent to the same destinations during the same window, producing a crawl-to-refer ratio. Published August 2025.
The Week Without ChatGPT Attribution (Evertune)
Evertune · Will Robinson · 2026
Evertune reviewed more than 25,000 ChatGPT responses between 15 September and 15 October 2026 and reports ChatGPT stopped appending utm_source=chatgpt.com to links from 9 October, with tags reappearing on 16 October. It cites Adobe data of about 31,000 fewer ChatGPT visits a day against about 26,000 more Direct visits across roughly 500 pages.
Methodology note
Vendor research post by Will Robinson, Evertune, published 16 October 2026 and updated 20 October. Samples 25,000 responses from repeated prompt runs; the traffic figures are Adobe's, relayed from Cedric Huesler. The Direct-versus-referral link is correlational, leaving a gap of about 5,000 visits a day. Read directly from the page.
ChatGPT Traffic Drop Appears To Be A Tracking Problem (Search Engine Journal)
Search Engine Journal · Matt G. Southern · 2026
From 9 October 2026, GA4 ChatGPT-attributed sessions fell 22% to 57% on sites tracked by Graphite, while Adobe's Cedric Huesler saw ChatGPT visits down about 31,000 a day and Direct up about 26,000 across roughly 500 pages. Profound tied the change to the GPT-6 Intelligent UI. Huesler says OpenAI confirmed a bug on 14 October and tagging began returning on 15 October.
Methodology note
Trade-press news roundup by Matt G. Southern, read directly. It compiles figures from Graphite, LLM Pulse, Adobe's Huesler (four days of sampled data), Profound and Peec AI, several without disclosed sample sizes. The page shows no absolute date, so 2026-10-15 is inferred from its content. Visitation loss versus relabelling is unconfirmed.
Do Not Force Search in Prompt Tracking (Graphite)
Graphite · Gregory Druck · 2026
Across 100 entity-comparison prompts with 100 logged-out ChatGPT responses per prompt in each setting (31 July to 5 August 2026), about 58% of unforced responses showed evidence of search. Forcing search changed answers, with the largest visibility differences on prompts that never trigger search naturally. In one test, prompts with no search in 600 default responses were classed as searched in all 18 forced responses.
Methodology note
Vendor research paper by Gregory Druck, PhD, Graphite, dated 20 August 2026 and read directly. Search use was inferred from fan-out queries or a Sources panel, not citations alone. Logged-out ChatGPT only, with a 10,872-response logged-in comparison from April 2026. Rates change often; the Similarweb 7% citation figure is cited secondhand.
Disentangling Answer Engine Optimization from Platform Growth: A Log-Based Natural Experiment on ChatGPT Referral Traffic (arXiv 2606.04362)
Keisuke Watanabe, Kazuki Nakayashiki · 2026
On one high-traffic domain (glasp.co), monthly ChatGPT referrals grew 5.7x while untreated pages on the same domain grew 3.5x, so most raw growth was platform tailwind. Treated pages gained an estimated further 1.82x (95% CI 1.31-2.54) after January 2026 AEO changes, but a placebo test gave p=0.16. The authors conclude headline AEO multiples overstate the causal effect.
Methodology note
arXiv preprint 2606.04362 by Keisuke Watanabe and Kazuki Nakayashiki, submitted 3 June 2026, not peer reviewed. Single domain; first-party analytics and server logs; treated pages compared with untreated pages on the same domain using an interrupted time series on the weekly ratio with HAC errors and a placebo-in-time test. Abstract page read; intervention details in section 4 were not read.
The 5-Layer Framework for Measuring GEO Performance (Search Engine Land)
Search Engine Land · Paul DeMott · 2026
An independently published five-layer GEO measurement framework: GA4 direct attribution, crawl-log diagnostics, share of voice plus structured AI interrogation, self-reported attribution form fields, and portfolio-level incrementality testing. It cites Cloudflare's June 2025 crawl-to-referral ratios of 1,700:1 for OpenAI and 73,000:1 for Anthropic, and the Loamly 70.6% Direct-misattribution figure.
Methodology note
Trade-press contributor article by Paul DeMott, published in Search Engine Land on 2026-05-18. A practitioner framework rather than a study: it presents no original dataset and sources its figures from Cloudflare and Loamly. Value here is independent convergence on the same measurement pillars. Fetched and read directly.
The Answer Economy: How AI Search Is Rewiring B2B Software Buying
G2 · Tim Sanders · 2026
G2's survey of 1,076 B2B software buyers, fielded March 2026, found 51% now start research with an AI chatbot more often than Google, up from 29% a year earlier. 69% chose a different vendor than planned on AI guidance, and a third bought from a vendor new to them. The top trust signal inside an AI answer is a citation from a software review site. Single-vendor survey with commercial interest.
Methodology note
First-party survey by G2 of 1,076 B2B software buyers and decision-makers plus 39 marketer interviews, fielded March 2026 and published 15 April 2026 as 'The Answer Economy', authored by Chief Innovation Officer Tim Sanders. G2 sells answer-engine-optimization products, so treat as directional, not independent. Verified against G2's own release and PR Newswire.
80% of Your AI Traffic Is Invisible. Here's What GA4 Misses. (Loamly)
Loamly · Marco Di Cesare · 2026
Loamly's 14,413 dark AI visits, 70.6% of 20,428 AI visits across 446,405 visits, are visits with no referrer that its own model scores as AI from paste-navigation, timing and behaviour signals above a confidence threshold. Only 100 of 8,874 ChatGPT visits were cryptographically verified. Dark AI was 3.2% of visits and 1.9% of visitors.
Methodology note
Vendor research post by Marco Di Cesare, founder of Loamly, published 3 February 2026 and read directly. Data spans all Loamly customer websites; the vendor states the base skews toward companies that care about AI traffic and is not a random sample. Classification signals are listed, but thresholds, validation and site count are not disclosed.
The AI Traffic Attribution Crisis: Why Your Analytics Are Wrong (Loamly)
Loamly · Marco Di Cesare · 2025
Across 446,405 visits in Loamly's database, 20,428 were classed as AI-sourced, and Loamly attributes 14,413 of those, or 70.6%, to AI despite no referrer header, so GA4 would file them as Direct. That classification is Loamly's own model inference (see R353), not an observed count. Named causes are copied URLs and ChatGPT mobile and Atlas referrer stripping.
Methodology note
Vendor analysis by Marco Di Cesare of Loamly, published 2025-11-07 and last updated 2026-02-16, drawing on the company's own visit database. Visit count and AI-visit split are disclosed; the number of distinct websites and the collection time window are not. Fetched and read directly.
AI Traffic Has Increased 9.7× in the Past Year (81,947 Websites Study)
Ahrefs · 2025
Across 81,947 websites, average AI traffic grew about 9.7 times in a year. The average site's search traffic dropped about 21% over the same period. AI traffic now represents 0.25% of a site's total traffic on average. ChatGPT grew 85% since January 2025 and now sends more traffic than Reddit or LinkedIn. Google still sends about 210 times more traffic than the big three AI platforms combined.
Methodology note
Ahrefs analysed referral traffic patterns across 81,947 websites between mid-2024 and mid-2025, comparing AI referrals (ChatGPT, Perplexity, Gemini, Copilot) against traditional search, social platforms, and direct traffic. The dataset more than doubled the size of the earlier March 2025 study.
How to find relevant prompts for your brand? (Otterly Help)
Otterly AI · 2024
Otterly's own help documentation explicitly states there is 'no way to learn which prompts are most asked at ChatGPT or Perplexity' and 'no way to know what exactly people are searching for in the AI engines.' Otterly recommends constructing prompts from available external inputs such as brand terms, domains, industries, URLs, and SEO keywords. This is a vendor admission that aligns with the public-proxy thesis.
Methodology note
Otterly AI Help Center article (last updated April 2026) describing the vendor's own recommended methodology for building a brand's prompt list. Self-reported vendor documentation; the page explicitly states that AI search engines do not publish query data and lists three substitute methods Otterly supports (Prompt Research tool, Google Search Console import, AI-assisted brainstorming). Content verified by direct fetch.
API vs UI - How to track LLM visibility?
Peec AI · Malte Landwehr · 2026
Across 555 prompts run 47 times each on both the ChatGPT UI and the API, 52,170 chats, the average brand's visibility differed by 41% between surfaces and the most visible UI brand missed the API top three. Competitor domains rose from 38% to 52% of citations, listicles fell from 25% to 4%, product pages and how-to guides grew from 34% to 63%, and API answers ran 31% longer.
Methodology note
Self-published LinkedIn article by Peec AI's CPO, 29 August 2026, read in full from a PDF capture. The design is large and repeated, and 47 runs per prompt per surface clears the sampling thresholds in R242 and R243, but the article discloses no temperature, no web-search setting, no UI account state and no GPT-5.6 variant, and reports no confidence intervals.
About the author Max Ackermann
Max Ackermann is founder and Managing Director of info.link, the product data platform that makes brands visible in AI search and connects every physical product to the web through GS1 Digital Link. He writes about AI search and generative engine optimization (GEO), AI-powered commerce, and how brands can structure product data for ChatGPT, Gemini, Perplexity, and retailer AI assistants like Amazon Rufus. For the past two years he has built the pipelines that put structured product data into AI answers, and run the experiments that test what actually moves AI citations.
Max has 20+ years of experience building digital products and businesses. He previously led McKinsey's Corporate Venture and Design teams across Europe, and as Managing Director of a leading US digital agency he built platforms with Nike, Google, Meta, and Airbnb. He founded the UX Design program at Central Saint Martins College, University of the Arts London, and is a Fellow of the UK's Higher Education Academy. Based in Hamburg, he works closely with GS1 on Digital Link adoption; info.link is headquartered in Hamburg and Berlin and counts GS1 Germany among its investors.
Follow Max on LinkedIn.
