---
title: "Your AI Visibility Score Is Quality Score With Worse Math"
description: "AI visibility dashboards turn a handful of changing answers into precise-looking scores. Here’s why I’d check a few buyer prompts repeatedly instead."
url: "https://groas.com/post/your-ai-visibility-score-is-quality-scor"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/5a55226a-4fcf-4af6-b31a-342ff218bc57.png"
published: "2026-10-15T05:33:53.109Z"
modified: "2026-10-15T05:33:53.168Z"
---

October 15, 2026 · 9 min read

# Your AI Visibility Score Is Quality Score With Worse Math

[Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

![A precision digital caliper reading 37.4 clamps a drifting cloud, mocking exact-looking scores that measure something shapeless.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/5a55226a-4fcf-4af6-b31a-342ff218bc57.png)

In this article

1. [Your 37.4% AI visibility score measures what, exactly?](#your-374-ai-visibility-score-measures-what-exactly)
2. [I watched Quality Score become a client presentation](#i-watched-quality-score-become-a-client-presentation)
3. [A few dozen prompts cannot stand in for a market](#a-few-dozen-prompts-cannot-stand-in-for-a-market)
4. [The answer changes when you ask again](#the-answer-changes-when-you-ask-again)
5. [Someone chose the prompt list](#someone-chose-the-prompt-list)
6. [A green arrow is not an action plan](#a-green-arrow-is-not-an-action-plan)
7. [The smaller test I would actually run](#the-smaller-test-i-would-actually-run)
8. [Three questions before you pay for the decimal](#three-questions-before-you-pay-for-the-decimal)

## Your 37.4% AI visibility score measures what, exactly?

**AI visibility scores are Quality Score with worse math.** Your brand is at 37.4%, up 2.1 points from last Tuesday, and nobody in the room can tell you what the denominator was. The number arrives on a slide or inside a $149-a-month dashboard, dressed in charcoal grey with a bright green arrow. You are supposed to nod at the decimal place and conclude that whatever your team published last week worked.

Maybe it did. The score cannot tell you.

Look under the hood and the arithmetic changes from tool to tool. As [technical teardowns of prompt tracking tools](https://queryburst.com/ai-visibility/prompt-tracker-scores/) show, Peec divides brand mentions across every prompt run; Profound excludes prompts where no brand was mentioned; Otterly plots a subjective two-dimensional index; and PromptWatch deducts points for competitive context. Put the same domain into three tools on the same morning and you can get three conflicting answers, each presented to a tenth of a percent.

That is not precision. It is a formatting choice.

## I watched Quality Score become a client presentation

I spent years managing Google Ads accounts through the spreadsheet era. In 2016, performance marketing had its own magical composite number: Quality Score. Agency decks celebrated a rise in average keyword scores from 6.2 to 7.1. Clients cheered, retainers renewed, and almost nobody asked whether closed-won pipeline had moved. [Google’s documentation](https://support.google.com/google-ads/answer/6167118?hl=en) describes Quality Score as a keyword-level, 1-to-10 diagnostic comparing ad quality with other advertisers. It is not an input into the ad auction or a measure of business profit.

I knew the number could swing while something more important went wrong. A client could change a landing page without telling the account manager. Cost per acquisition could climb 40% while the deck still displayed an 8-out-of-10 score. That score made a convenient answer when the harder work was checking negative lists, conversion tracking, and whether a bid cap was starving high-intent auctions.

**An AI visibility score offers the same refuge:** a technical-looking proxy that delays the question of whether the brand made any money. The difference is that Quality Score at least referred to a defined keyword diagnostic. Here, the prompt list and even the denominator can change depending on who sold you the dashboard.

## A few dozen prompts cannot stand in for a market

Traditional paid-search share of voice had an auditable denominator: Google reported auction impressions for a targeted keyword list and the share your ads captured. Generative engines have no equivalent fixed inventory. As [Search Engine Land noted about AI share of voice](https://searchengineland.com/ai-share-of-voice-metrics-that-matter-more-479611), a percentage share of an open-ended conversational universe is a different proposition. A buyer might ask about migration timelines from HubSpot, add a European data-residency constraint, then request a shortlist. That is not one static keyword with one static results page.

Yet a tracker may run 25 to 50 fixed prompts once a week, count mentions, and present the result as your market standing. **Fifty synthetic queries on Tuesday tell you how the engines answered those fifty sentences on Tuesday.** They do not tell you what every buyer sees.

If a competitor appears on 18 prompts this week and 19 next week, a chart can turn that one extra appearance into a 2.0-point surge. Cue two hours in an all-hands meeting about the breakout. The engine may simply have pulled from a different cached index.

![Editorial illustration of a brass gauge measuring one teacup while an ocean of conversational queries stretches behind it.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/2c55dbec-f0b4-4ff1-907d-e0821ade0929.png)

## The answer changes when you ask again

Generative answers are not stable ranking positions. Even when an identical prompt is run under tightly controlled conditions, the wording can change: an [experiment documented by Promptly](https://www.promptly.fyi/blog/why-ai-gives-different-answers) produced 80 distinct completions from 1,000 runs at temperature 0. In a [90-day benchmark of eight AI search engines](https://maxaeo.ai/blog/ai-answer-volatility-study/), identical buyer prompts kept the same top-three recommended brand set only 46% of the time from run to run.

So a weekly ping is a snapshot of a moving answer, not a permanent ranking. Ask again and your brand may disappear from the summary. A rise from 34% to 41% could reflect a change worth investigating, or it could be ordinary variation. **One run cannot distinguish the two.** Measuring wind speed once on Monday and calling it the quarter’s climate would look ridiculous. Put the result in a polished dashboard and apparently we need a meeting.

Blending engines makes the number murkier. ChatGPT, Google AI Overviews, Claude, and Gemini do not draw on identical systems. [Cross-model brand visibility research](https://searchengineland.com/ai-visibility-index-brands-vanishing-from-ai-search-485057) found that only 11% of tracked brands appeared across all three major models, while 77% appeared in only one. Notion was cited 17 times more often in Claude than in Gemini. If your brand appears in Claude but not Google AI Overviews, a blended 37% conceals both facts. That is the information you needed.

## Someone chose the prompt list

Before trusting a share-of-voice chart, ask where its prompts came from. An agency might brainstorm forty conversational queries; a tool might use your sitemap to generate a list of ‘typical commercial search prompts’. Neither method establishes that a buyer with procurement authority asks those questions.

The list can also flatter the client by design. Mix twenty prompts about your specific features with ten generic queries and, unsurprisingly, the brand shows up. **That is confirmation bias with an API bill.** In paid search, nobody should buy exact-match keywords for a client’s own name and call the resulting impression share proof of market-wide demand. Dressing self-selected AI prompts in an executive dashboard does not improve the trick.

The prompt list matters because it decides what the score can see. Leave out the question a buyer asks two days before signing, and you can have a beautiful upward line while losing the answer that mattered.

## A green arrow is not an action plan

Say your score falls from 38.2% to 34.1%. What do you do before lunch? A discussion on [r/SEO_tools_reviews](https://www.reddit.com/r/SEO_tools_reviews/comments/1vbnlkz/do_u_actually_use_ai_search_visibility_scores_to/) describes composite visibility scores and share-of-voice charts as more useful for client slides than for deciding what to change. I can see why. The drop does not tell you whether a competitor published a useful teardown, an engine refreshed its index, forum sentiment changed, or that week’s run produced a different answer.

![Pen-and-ink cartoon of an agency presenter pointing at a rising green arrow while clients face an empty whiteboard labelled Action Items.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/c4d72a04-3ed7-4637-9221-aaa47da8a323.png)

**You cannot fix an undifferentiated composite.** If an agency cannot identify what changed, it cannot tell you what to build, rewrite, or repair. You get ‘build entity authority’ or ‘expand thought leadership’, which is what a recommendation sounds like when nobody wants to name a page. If that pitch sounds familiar, read [this open letter to the owner about to buy an AI visibility dashboard](https://groas.com/post/an-open-letter-to-the-owner-about-to-buy) before cutting a cheque.

A dashboard can flag a symptom. It cannot fix a blocked crawler, update a page, or adjust a bid. Paying to watch the weather change is still paying.

## The smaller test I would actually run

### Five buyer prompts, repeated rather than multiplied

I would start with **five to ten questions real buyers ask close to a demo or contract**, with the commercial intent and constraints left intact. Run the same prompts across the engines that matter to your business, then sample each prompt 10 to 20 times across different days and hours. Do not turn one Tuesday answer into a market-share claim.

We set out a broader version of this approach in [the 30-prompt test for AI search visibility](https://groas.com/post/the-30-prompt-test-measure-your-brand-s). The point is not that thirty is a magic number. Repeated runs show whether a brand appears reliably for a useful question; a single weekly run shows what happened once. I would rather know the answer to five buyer questions repeatedly than admire one composite score built from 200 prompts I cannot defend.

### Count mentions, citations, and recommendations separately

A tool can register a ‘win’ whenever an assistant includes your name somewhere in a long answer. That collapses three different events into one numerator:

- **Mention:** Your brand appears as item eight in a list of twelve vendors, without a link, context, or endorsement.
- **Citation:** The engine points to your site as a source and provides a clickable link that a buyer can follow to check details.
- **Recommendation:** The answer weighs the buyer’s constraints and presents your business as the preferred choice.

![Infographic comparing three tiers of AI search presence: mention, citation, and recommendation.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/660ebedd-2563-4e4c-a6e7-62633eefcbdf.png)

An unlinked name-drop and a linked recommendation should not count as the same outcome. **Separate the three, or the headline number will hide the useful part.** Our guide to [building an AI visibility tracker](https://groas.com/post/build-your-own-ai-visibility-tracker-in) covers measuring mention rates alongside citation rates rather than saving full-page screenshots. A recommendation on a core commercial prompt tells me more than a long list of raw mentions on informational ones.

### Check what changed before claiming a win

A citation-rate drop in ChatGPT means little without a change log. Did someone update robots.txt and block GPTBot? Did marketing rewrite a comparison page and remove structured pricing data? Did a competitor publish a technical teardown? The log cannot explain every swing, but without it you cannot even begin to separate an operational cause from answer variation.

This is the fix I want: **connect monitoring to execution, not another reporting meeting.** With [groas](https://groas.com/), specialized AI models work across paid and organic search continuously, including technical adjustments, landing-page changes, and bids. Actions and their reasoning are logged, while a named human strategist sets commercial guardrails. That is a better operating model than paying an agency to narrate a monthly chart: you can inspect what changed and judge the work against attributable revenue.

## Three questions before you pay for the decimal

Before buying an AI visibility subscription or approving an agency’s monthly ‘AI Share of Voice’ slide, I would ask:

1. **What is the denominator, and how often was each prompt sampled?** If the vendor cannot explain its formula or ran each prompt only once that week, the decimal place is doing more work than the sample.
2. **Do you distinguish a mention from a citation and a recommendation?** A passing name-drop in ten options does not carry the same commercial weight as a linked answer that recommends you for the buyer’s constraints.
3. **When the number moves, what executes the fix?** A chart can flag something to investigate. A monthly recommendation meeting cannot, by itself, repair a page, resolve a crawl block, or change an auction bid.

Founders sometimes ask me which software will automate AI search visibility reporting without hiring an agency. I think that automates the wrong layer. A faster chart of answers you sampled once does not make those answers more reliable, and it certainly does not make the next change for you. I want repeated checks on buyer prompts, a record of what changed, and execution accountable to pipeline rather than cosmetic percentages.

**The decimal places reset when you refresh the page. Closed-won revenue does not.**

## Frequently Asked Questions

### Why do different AI visibility tools give different scores for the same brand?

Each tool calculates the score with its own arithmetic. Peec divides brand mentions across every prompt run, Profound excludes prompts where no brand was mentioned, Otterly plots a subjective two-dimensional index, and PromptWatch deducts points for competitive context. The same domain checked in three tools on the same morning can produce three conflicting answers.

### Is an AI visibility score a reliable measure of business results?

No. Like Google Ads Quality Score, it is a technical-looking proxy that can stay healthy while more important things go wrong. The article notes Quality Score could show 8 out of 10 while cost per acquisition climbed 40%, and it was at least a defined keyword diagnostic, whereas AI visibility scores can have shifting prompt lists and denominators.

### Do AI visibility trackers really measure your market share?

No. A tracker may run 25 to 50 fixed prompts once a week and count mentions, but generative engines have no fixed inventory like paid-search auction impressions. Fifty synthetic queries on a Tuesday only tell you how the engines answered those fifty sentences that day, not what every buyer sees.

### Are AI answers the same every time you run the same prompt?

No. Even under tightly controlled conditions, answers vary: one experiment produced 80 distinct completions from 1,000 runs at temperature 0, and a 90-day benchmark of eight AI search engines found identical buyer prompts kept the same top-three brand set only 46% of the time. A weekly run is a snapshot of a moving answer, and one run cannot tell whether a score change is real or ordinary variation.

### Do AI visibility scores hide differences between ChatGPT, Claude and Gemini?

Yes, blending engines into one number conceals engine-specific results. The engines do not draw on identical systems: research found only 11% of tracked brands appeared across all three major models while 77% appeared in only one, and Notion was cited 17 times more often in Claude than in Gemini. A blended 37% hides whether you appear in Claude but not Google AI Overviews.

### How can a prompt list make an AI visibility score look better than it is?

Whoever builds the list can pick prompts that favour the client, for example mixing twenty prompts about the brand's own specific features with ten generic queries, which makes the brand show up. This is confirmation bias, and if the list leaves out the question a buyer asks two days before signing, the score can rise while you lose the answer that mattered.

### What should you do when your AI visibility score drops?

A composite score drop does not tell you what changed, so you cannot fix it directly. The decline could come from a competitor publishing a teardown, an engine refreshing its index, changed forum sentiment, or just a different answer that run. Instead of acting on the chart, check what actually changed, such as a blocked crawler or an edited page.

### How should I measure AI search visibility instead of using a dashboard score?

Use five to ten questions real buyers ask close to a demo or contract, keeping commercial intent and constraints intact. Run the same prompts across the engines that matter to your business and sample each prompt 10 to 20 times across different days and hours. Repeated runs show whether your brand appears reliably, which one weekly run cannot.

## Pay For Results, Not For Hours

Businesses buy the outcome, agencies resell it, and groas answers for it either way.

[See If You Qualify](https://groas.typeform.com/to/xC1bQNUT)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/your-ai-visibility-score-is-quality-scor#webpage","url":"https://groas.com/post/your-ai-visibility-score-is-quality-scor","name":"Your AI Visibility Score Is Quality Score With Worse Math","description":"AI visibility dashboards turn a handful of changing answers into precise-looking scores. Here’s why I’d check a few buyer prompts repeatedly instead.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/5a55226a-4fcf-4af6-b31a-342ff218bc57.png"},"breadcrumb":{"@id":"https://groas.com/post/your-ai-visibility-score-is-quality-scor#breadcrumb"},"dateModified":"2026-10-15T05:33:53.168Z","mainEntity":{"@id":"https://groas.com/post/your-ai-visibility-score-is-quality-scor#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/your-ai-visibility-score-is-quality-scor#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"Your AI Visibility Score Is Quality Score With Worse Math","item":"https://groas.com/post/your-ai-visibility-score-is-quality-scor"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/your-ai-visibility-score-is-quality-scor#article","url":"https://groas.com/post/your-ai-visibility-score-is-quality-scor","mainEntityOfPage":{"@id":"https://groas.com/post/your-ai-visibility-score-is-quality-scor#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"Your AI Visibility Score Is Quality Score With Worse Math","description":"AI visibility dashboards turn a handful of changing answers into precise-looking scores. Here’s why I’d check a few buyer prompts repeatedly instead.","datePublished":"2026-10-15T05:33:53.109Z","dateModified":"2026-10-15T05:33:53.168Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/5a55226a-4fcf-4af6-b31a-342ff218bc57.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"score, visibility, quality score, prompts, quality, cannot, visibility score, buyer","wordCount":1775,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"Ex Goldman Sachs and Ex Stanford Computer Science","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/your-ai-visibility-score-is-quality-scor#faq","mainEntity":[{"@type":"Question","name":"Why do different AI visibility tools give different scores for the same brand?","acceptedAnswer":{"@type":"Answer","text":"Each tool calculates the score with its own arithmetic. Peec divides brand mentions across every prompt run, Profound excludes prompts where no brand was mentioned, Otterly plots a subjective two-dimensional index, and PromptWatch deducts points for competitive context. The same domain checked in three tools on the same morning can produce three conflicting answers."}},{"@type":"Question","name":"Is an AI visibility score a reliable measure of business results?","acceptedAnswer":{"@type":"Answer","text":"No. Like Google Ads Quality Score, it is a technical-looking proxy that can stay healthy while more important things go wrong. The article notes Quality Score could show 8 out of 10 while cost per acquisition climbed 40%, and it was at least a defined keyword diagnostic, whereas AI visibility scores can have shifting prompt lists and denominators."}},{"@type":"Question","name":"Do AI visibility trackers really measure your market share?","acceptedAnswer":{"@type":"Answer","text":"No. A tracker may run 25 to 50 fixed prompts once a week and count mentions, but generative engines have no fixed inventory like paid-search auction impressions. Fifty synthetic queries on a Tuesday only tell you how the engines answered those fifty sentences that day, not what every buyer sees."}},{"@type":"Question","name":"Are AI answers the same every time you run the same prompt?","acceptedAnswer":{"@type":"Answer","text":"No. Even under tightly controlled conditions, answers vary: one experiment produced 80 distinct completions from 1,000 runs at temperature 0, and a 90-day benchmark of eight AI search engines found identical buyer prompts kept the same top-three brand set only 46% of the time. A weekly run is a snapshot of a moving answer, and one run cannot tell whether a score change is real or ordinary variation."}},{"@type":"Question","name":"Do AI visibility scores hide differences between ChatGPT, Claude and Gemini?","acceptedAnswer":{"@type":"Answer","text":"Yes, blending engines into one number conceals engine-specific results. The engines do not draw on identical systems: research found only 11% of tracked brands appeared across all three major models while 77% appeared in only one, and Notion was cited 17 times more often in Claude than in Gemini. A blended 37% hides whether you appear in Claude but not Google AI Overviews."}},{"@type":"Question","name":"How can a prompt list make an AI visibility score look better than it is?","acceptedAnswer":{"@type":"Answer","text":"Whoever builds the list can pick prompts that favour the client, for example mixing twenty prompts about the brand's own specific features with ten generic queries, which makes the brand show up. This is confirmation bias, and if the list leaves out the question a buyer asks two days before signing, the score can rise while you lose the answer that mattered."}},{"@type":"Question","name":"What should you do when your AI visibility score drops?","acceptedAnswer":{"@type":"Answer","text":"A composite score drop does not tell you what changed, so you cannot fix it directly. The decline could come from a competitor publishing a teardown, an engine refreshing its index, changed forum sentiment, or just a different answer that run. Instead of acting on the chart, check what actually changed, such as a blocked crawler or an edited page."}},{"@type":"Question","name":"How should I measure AI search visibility instead of using a dashboard score?","acceptedAnswer":{"@type":"Answer","text":"Use five to ten questions real buyers ask close to a demo or contract, keeping commercial intent and constraints intact. Run the same prompts across the engines that matter to your business and sample each prompt 10 to 20 times across different days and hours. Repeated runs show whether your brand appears reliably, which one weekly run cannot."}}]}]}
```