Every testing programme has more ideas than traffic. Choosing which to run first matters more than how many you run. The most reliable predictor of a winning test isn't how good the idea sounds; it's the evidence behind it.
Score every idea on four things
- Evidence: is there research, data or customer feedback showing a problem? Ideas from opinion score lowest.
- Reach: how many visitors reach the page or step? More traffic means faster, more sensitive tests.
- Effort: design, build and QA time.
- Confidence: how sure you are the change addresses the problem.
Weight evidence most
Frameworks like ICE or PIE are useful, but they often weigh a hunch the same as a funnel analysis. Weighting evidence more heavily pushes well-founded ideas up and opinions down. Over time, check which evidence sources produced winners and adjust the weights.
Check feasibility before scoring
A great idea on a page that gets 500 visitors a week can't be tested properly. Before scoring, check the traffic can detect the effect you'd expect in a reasonable time. If not, the idea needs research, a bigger change or a different method.
Keep the backlog healthy
- Review it monthly. Re-score ideas as new evidence arrives.
- Retire stale ideas. If nobody can explain the evidence, remove it.
- Balance quick wins and bigger bets. A programme of only small tweaks rarely finds large gains.
Make it visible
Share the ranked backlog with stakeholders. It turns "why isn't my idea being tested?" into a discussion about evidence, and it makes the case for research when the backlog is full of guesses.
Prioritising by evidence produces fewer tests that teach more, and a higher win rate over time.
