2/6/2021
Published in UX Collective (uxdesign.cc).
The piece starts from a client who needed to explain the benefits of experimentation to their company. It argues that the main problem isn’t just “we should test”—it’s that many teams don’t see the limits of what they do today: user research (small, unrepresentative samples; say vs. do), pre/post analysis (changing environment, many confounds), and releasing on gut feel (typical win rates ~20–30%, so most ideas don’t improve conversion).
It then defines what an experiment is (scientific, controlled, repeatable), outlines the experiment life cycle (hypothesis → design → run → compare), and describes three experiment types by goal: learning about users, increasing conversion, and de-risking changes. Benefits of a single experiment: better insights, testing assumptions, and de-risking. Benefits of an experiment program (multiple experiments, built on something like Kolb’s learning cycle): more objective decisions, consistent processes (e.g. structured A/B testing), and better ROI. The article ties this to the “Widerfunnel infinity” style of process and ends with competition and culture.
