Previous
¶ Experimentation Plan Iteration
¶ What it is
In a typical Validation process, multiple bouts of experimentation will be run. Typically, after the first set of experiments concludes and we see the outcomes, there might be causes to iterate on our experimentation plan. This can include changing the order of the following experiments based on the learning or modifying future experiments because of learning from a round of experimentation.
¶ Why it is useful
Experimentation should be run as an agile process in which we regularly adjust our activities to the reality we are confronted with to ensure the utility of all activities we perform. Learning from bouts of experimentation to adjust the future plan ensures we do not just execute a plan that stopped being useful but actually continues to improve the utility of our plan and actions.
¶ When to use it
Whenever a team runs multiple rounds of experimentation, it should take a reflection step between the various rounds and adjust its plans as needed.
¶
¶ How is it done
- After a set of experiments is concluded and before the next set starts, set up a workshop session with the goal of reflection and improvement.
- Talk through all experiments and outcomes, and discuss with the team what this means for the next round:
- Certain experiment types might have underperformed, and we want to use different experiment forms in the future.
- Specific experiments might have yielded data that invalidates an assumption, and we don’t need to continue validating assumptions that depend on that one and instead focus elsewhere.
- Specific experiments might have been inconclusive and need a new experiment designed to be run.
- After all, implications are discussed, review the Experimentation Board and change the setup for the next round of experiments.
¶ Do's & Don't
Do's
- Take a break between two sprints of experimentation to do a retrospective and improve your planning for the next round.
- Challenge your plan critically and ensure you are learning fast enough and deep enough to make the progress you want to see.
Don't
- Don’t change the plan just because you disagree with the data or don’t like the data - these are results, not mistakes.
- Don’t iterate the concept at this point unless you invalidated a “Make or Break” assumption that changes everything - in which case, move to “Concept Review” to make fundamental changes.
¶ Tools needed
- Whiteboard (physical or virtual)
- Post-it notes (physical or virtual)
- Experimentation Board
¶ Example
