Reliability
How much should I trust this number?
Example finding
Catch a noisy prompt before it misleads you
A prompt that names you on two of three repeats shows up with its flip rate and a link to all three answers, instead of blending quietly into one average.
What you see
Every headline rate shown with a 95% interval (Wilson score, widened by a prompt cluster bootstrap), repeat-run agreement when a prompt was asked more than once, and whether the rate has held steady across your last several audits.
Reliability snapshot
Sample size, 95% interval and a counts-only, low or adequate label for every headline metric.
Repeat-run agreement
When the same prompt and engine pair ran more than once, how often the repeats agreed on whether you were named.
Stability across audits
Whether a rate's mean and range have stayed steady across your last several audits, or swung more than normal variation.
Flipped answers
The exact prompt and engine pairs that disagreed across repeats, with a link to every answer.
How it works
- 1
Interval
A Wilson score interval is computed for every rate, then widened to the wider of that and a bootstrap that resamples whole prompts, so repeated runs of one prompt are never read as independent evidence.
- 2
Agree
Prompt and engine pairs run more than once are grouped, and the share that agree with the majority answer is reported alongside the flip rate.
- 3
Track
Each audit's rate is read from stored metric snapshots. Once three or more audits have enough runs, the mean, range and a stable or variable label are computed from them.
How it is measured
Definitions come first, so numbers stay comparable from one audit to the next.
- 95% interval
- Wilson score interval, widened by a prompt cluster bootstrap when the sample supports one.
- Flip rate
- Share of repeated prompt and engine pairs where the repeats disagreed on whether the brand was named at all.
- Stability label
- Stable or variable, from the coefficient of variation of a rate across at least three audits with enough runs each.
Good to know. Reliability describes the sample you have, it does not create more of it. Below five completed runs a rate is shown as a count, never a percentage, and stability needs at least three audits with enough runs in each.
Common questions
Why does a rate sometimes show as a count instead of a percentage?
Below five completed runs a percentage would overstate how much is known, so the raw count is shown instead.
What counts as a repeat?
The same prompt asked of the same engine more than once within one audit. Repeats across different engines are not compared to each other.
See Reliability on your own brand.
Start with a first audit. We show where you appear, who is named instead, and what to fix first.