How long should a test run before you trust the result?
- Published November 6, 2026
- Linked from 1 answer
- In a cluster of 77
Until three conditions hold, whichever takes longest: Google has crawled every page in the test since you changed it, the test has covered whole weeks, so weekday and weekend patterns are both in the data, and the difference you see is larger than the week-to-week movement the same pages show when nothing changes. A fixed number of days cannot stand in for those conditions, because a week is plenty on a page with heavy traffic and far too little on a quiet one.
Three conditions before you read the result
| Condition | Why it matters | How to check |
|---|---|---|
| The change has been crawled | Until Google fetches the new version, it is still ranking the old one | URL Inspection shows the last crawl date for each test page |
| Full weekly cycles are covered | Traffic on weekdays and weekends can differ enough to fake an effect | Count whole weeks from the date the last test page was crawled |
| The effect beats normal noise | A gap smaller than ordinary weekly variation is not a result | Compare against an unchanged control group, or against the same pages' movement in prior weeks |
Search Console data also arrives with a delay of a few days, so the last days of any window are incomplete. Read the test once the data for its final day has filled in.
Use a control group
The cleanest way to separate your change from everything else is to compare changed pages with similar pages you left alone, over the same dates. An update, a seasonal shift, or a change in demand can move both groups, while your change moves only one, but only if the control pages are a close match: the same template, similar queries, similar traffic. Check that the two groups moved together in the weeks before the change; if they already drifted apart then, the comparison will not isolate your edit. Even a good control gives an estimate, not proof. Without a control, a test that runs during a core update measures the update, not your edit.
When the test cannot reach a verdict
Low-traffic pages may never produce a difference larger than their noise, however long the test runs. In that case the correct result is “inconclusive”, and the choice is to test on a larger group of similar pages or to decide on other grounds. Judging a single change to one page, rather than a controlled test, is a different question with its own answer in how long to wait before judging whether a change helped. Write down the metric and the end date before the test starts: stopping a test on the first day the numbers look good is how a result that later disappears gets reported as a win.
Add a comment