Running Product Experiments: Best Practices to Drive Results
This article contains 10 expert tips for product managers on how best to run experiments in ways that drive product growth
On running experiments, I have increasingly observed that some product organizations still plan budgeting before de-risking their vision, timing and execution. Naturally to make better decisions whilst building products, two critical places to begin are: talking to your potential customers and running experiments.
Experiments- though not a complete substitute for product research- are pivotal tools to help your team learn by doing, catching bells and whistles you missed during the research stage, and updating your knowledge of what the customer will do in real-usage use cases.
I have shared below 10 world class tips essential for running successful experiments:
- Experiment with your product, but don’t stop there. Analyze your customer segments, value propositions, customer acquisition methods, pricing plans, revenue models, and unit economics.
- Explore rigorous thinking at the start because you can test anything in more ways than you think.
- It is imperative that you pay attention to the details because poorly planned experiments result in sloppy results.
4. Build your experiments in a way that allows you to learn just as much, if not more, with half the time and effort.
5. Launch your experiment with the mindset of changing customer behavior. If your A/B test does not cause customers to change a pattern, it was not successful. In other words, your customers should not be lukewarm towards a (helpful) feature.
6. You will often discover things to improve by doing a trial-run before running large scale experiments.
7. Be intense and fast when running your experiments because time will disappear much faster than you expect.
8. Take advantage of qualitative research opportunities (by talking to your users!) during the process.
9. Be aware of your confirmation biases. To summarize, don’t twist the results to fit your agenda, or dismiss unfavorable results too quickly.
10. To make sound product decisions, merge evidence and judgment (product sense/intuition) and execute!
