Previous
¶ Metrics Definition
¶ What it is
Metrics Definition is the creation of a preliminary plan of what data you wish to track and how data will perform. It is the first step to validate whether the data performance of new products is within expectation to decide the next steps.
¶ Why it is useful
What data to track and monitor is quite standard as it should fit into company goals, i.e., revenue and orders. However, process indicators are also crucial besides outcome indicators, especially when the results are not as expected. Thus, data prediction will contain business outcome indicators and more concrete indicators such as product usage and conversion, which requires data collection that we will introduce in the next phase.
¶ When to use it & Who is involved
Business analysts make data predictions when reviewing with product owners to make sure they're putting effort into products that have the potential to contribute to business value.
¶ How is it done
- Before your product launch, convene a workshop with the business, design, and technical teams.
- Discuss the key business indicators for product performance. What are the outcomes we want?
- Name the relevant indicators (such as revenue, repeat orders, etc.)
- Assign target values of time to indicate the expected outcomes and trend lines.
- Review the business indicators and build a metric tree for each, which comprises the metrics that lead to this outcome.
- e.g. Monthly Revenue = Number of Customers x Average Times Ordered per Month x Average Basket Size
- Break it down further from there until you get to concrete product flow metrics, e.g., new sign-up conversion and conversion of products viewed to products in the basket to products purchased
- Discuss how to measure these process indicators and what values suggest average, under-, and over-performance versus plan.
¶ Do's & Don't
Do's
- Think holistically about the customer journey and which behaviours lead to which outcomes.
- Discuss customer behaviour level and desired behaviour based on the product definition and critical strategic decisions.
Don't
- Don’t just look at outcome metrics. It is essential to consider leading indicators which lead to outcomes later.
- Don’t stop drilling down until you have single, measurable data points that the product team can turn into analytics that can be delivered to the business team regularly.
¶ Tools needed
- Virtual whiteboard to capture notes
- Spreadsheets to build data models
¶ Example
¶ Data Collection
¶ What it is
Data collection is the process of implementing the data tracking plan according to the defined metrics to generate the relevant data required for review.
¶ Why it is useful
Acting on our data needs means we have relevant data points (quantitative and qualitative) available when discussing product changes or strategy changes. Decisions based on data will yield better results than intuition or pure opinion.
¶ When to use it
Every product launched in the market should be set up to collect critical metrics for business review to ensure decisions can be made data-driven.
¶ How is it done
- Depending on the metrics you defined, implement the relevant tracking and collection mechanisms:
- Product analytics
- NPS & Customer feedback
- Customer interview cycles
- Marketing effectiveness tracking
- Ensure the data collection is as automated as possible and raw data is transformed into relevant information in a standardised form.
- Regularly review the collected data for quality and quantity and the coverage of the critical production processes. Optimise your tracking over time by covering more aspects of the customer journey in automated ways.
¶ Do's & Don't
Do's
- Automate, automate, automate. Manual data collection and analysis are more frustrating to team members.
- Write scripts that create a complete output like a marketing KPI analysis with key indicators and data visualisation. This will make business discussions much easier.
Don't
- Don’t just rely on off-the-shelf analytics. If you can’t find a tool to suit your needs, implement the relevant tracking.
¶ Tools needed
- Different analytics software to cover all the relevant needs
¶ Example
