How to check whether an action worked

After you publish a change, the honest question is whether the measure you targeted moved. Verification answers that with the same method you used to find the problem.

Steps

1. 1Record the measure and the baseline before the change goes live.
2. 2Publish the change and note the date.
3. 3Wait. AI platforms take time to discover and use changes, and the time varies.
4. 4Rerun the same observation set on the same platforms.
5. 5Compare against the baseline with sample sizes shown.
6. 6Mark the result as verified, inconclusive or not yet observable.

Avoiding false conclusions

• Compare like with like: identical prompts and platforms.
• Know your normal run to run variation before calling a change real.
• Report association, not proof, unless you have ruled out other explanations.

Why verification is the step teams skip

Finding a gap and making a change feel like the work. Checking whether the change did anything feels like an extra. It is the step that turns activity into learning. Without it, a team cannot tell which of its efforts are worth repeating, and reports end up listing work done rather than results achieved.

Define the measure before you act

Verification only works if the target was defined in advance. Decide which rate should move, on which prompts and platforms, and by how much you would consider meaningful. Record the baseline and the number of observations behind it. If you decide after the fact what counts as success, you will find some.

Know your normal variation

Before calling a change real, know how much the same measurement moves when nothing has changed. Run the untouched prompts twice a few days apart and note the difference. A movement inside that range is noise. This one habit prevents most false conclusions.

Three outcomes, and what each means

Association, not proof

Even a verified result is best described as an association. The model, the retrieval index, competitors' pages and seasonal interest can all change at once. A careful report says the measure moved after the change, shows the evidence, and states what else could have contributed. It only claims cause where the design rules the alternatives out.

Timing

Time to effect varies. Where an assistant retrieves live pages, a corrected page can show up within days. Where an answer relies on training knowledge, a change may take far longer, if it appears at all. Crawler settings are also not instantaneous: OpenAI documents that changes to robots.txt rules for its search crawler take about 24 hours to process. Set the rerun date accordingly and repeat it more than once.