Assumption Backlog & Experimentation Board
PDF Template
Assumption Backlog & Experimentation
Board Miro TemplatePrevious
¶ Experiment Design & Selection
¶ What it is
Now that we have a clear view of our assumptions, we need to find ways of testing them, to evaluate whether these assumptions are correct. This is where experiment design comes in. In the Experiment Selection, we want to imagine different ways of getting the data needed to evaluate the assumption and then decide on the most suitable experiment for our Validation phase.
¶ Why it is useful
Validation is all about designing and running experiments to get real-world data on the hidden assumptions that risks that might kill our ideas out in the real world. Designing good experiments is the foundation of getting the correct data.
¶ When to use it
Whenever you are looking to run an experimentation process, it should be rooted in a strong experiment design process, to inform it.
¶
¶ How is it done
- Prepare a workshop board with the Assumption Backlog boards from the Assumption Mapping exercise.
- Start the session by recapping what types of experiments there are and what ways there are of cheaply generating market data and insights.
- Go through each of the assumptions with the team, asking what the fastest way to get some data is and what is the way to the highest quality of data available to address any given assumption. Answer these two questions first for each hypothesis, and ignore the third question for now.
- After you have worked through the first two questions for each assumption, and we have a good idea of the experiments that might be relevant for us, go through each concept to answer the question of which experiment you will run based on the discussion. Again, try to find points of leverage where experiments can answer multiple questions (as long as this does not introduce unnecessary complexity or bias).
¶ Do's & Don't
Do's
- Keep in mind Effort-Insight-Fit - try to expand the least amount of effort in an experiment you choose for a given insight you want to generate. Experiments are throwaway work and will not be built on.
- Try to incorporate multiple experiments that give you different kinds of data (first-hand qualitative, first-hand quantitative, second-hand research, etc.) to avoid getting biased by single types of data.
Don't
- Don’t always go for the fastest or highest-quality way of generating data; try to understand the impact of the assumption, and the effort of a given method, to appropriately select what to go for.
¶ Tools needed
- Whiteboard (physical or virtual)
- Post-it notes (physical or virtual)
- Assumption Backlog Template
¶
¶ Example
¶
¶ Experiment Design & Selection Resources
¶ GenAI Tool Use Case
Please refer to Concept Critique: Suggestion on Designing, Prioritizing, or Reframing Concepts