9/6/2024
Published on Experiment Nation.
Hosted by Charlotte Bomford. 6 September 2024.
Iqbal Ali joins Experiment Nation to talk about solution bias: the habit of latching onto a solution before the underlying problem is clear, so the solution often isn’t tied to a real user or business problem.
What we covered
- Why problem discovery comes first. A strong experiment program starts with identifying and validating problems—from user research, reviews, surveys, feedback, and analytics—and separating user problems from business problems.
- What solution bias is. We’re drawn to solutions quickly; the “solution” gets decided with little or no link to the problem, and carries a lot of extra baggage. AI can help by keeping that connective tissue between problem and solution.
- Human–AI interaction pattern. A playbook (with Craig Johan and Marcela Sullivan): load your data (reviews, voice-of-customer, etc.), prime the AI, then use a structured pattern to explore the problem (probe deep and wide, avoid solutionizing). Only after that do you move to ideation and problem statements. It’s a back-and-forth: you prompt, AI responds, you reflect and prompt again.
- Benefits. Using AI this way makes solution bias visible, diversifies thinking, builds empathy with users, raises the bar in workshops, and lets AI suggest “crazy” ideas so humans can build on them without feeling exposed.
- Dealing with resistance. Frame workshops as low-stakes and fun; let people see their own solution bias through the exercise. For those reluctant to use AI, invite them to observe—often that’s enough for an “aha” moment. Position AI as an intern: you stay in control; human + AI together beat either alone.
- Minimising problems before, during, and after tests. Use a premortem: predict outcomes (win, inconclusive, loss, data/SRM issues), work back to possible causes, then plan mitigations, metrics, pilot tests, or A/A tests. Accept that experiments sometimes reveal a different problem than the one you aimed at—and realign when that happens.
- Takeaway. Skipping problem exploration is a mistake. The link between business problem, user problem, problem statement, and solution has to be explicit; otherwise you’re “shooting without a target.”
