Answer engines are non-deterministic: the same prompt can return different brands on consecutive runs. Any test design that assumes a stable response will produce confident nonsense. The fix is repetition, sample each prompt several times per reading period and compare rates, never single outcomes.
Change one element at a time. If you rewrite the persona and the disqualifier in the same week you cannot attribute the movement, and the temptation to credit the more interesting change is overwhelming. Version every hint with a date and keep the previous text; a hint library without history cannot be learned from.
Give each test a floor before you read it. As a rule of thumb, at least twenty sampled responses per prompt per variant before you draw any conclusion, and a full two-week reading window before you act on a change in a mature account. Shorter reads mostly measure engine drift.
Watch the right outcome. Presence is the coarsest signal; first-named rate and framing are where real movement shows up first. A hint change that moves you from third-named to first-named is a large win that a presence-only report will render as no change at all.
Separate engine drift from your own effect by keeping a holdout: a handful of clusters whose hints you deliberately leave untouched. When the holdout moves, the market moved. Without a holdout, every industry-wide shift gets misread as the result of your last edit.
Write down the hypothesis before the test. One sentence: "Adding a seat-count ceiling to the disqualifier will raise first-named rate on the SMB cluster without reducing presence on the mid-market cluster." Tests without a written hypothesis reliably turn into stories told after the fact.