Last week, I discussed Go-to-Market and Organizational Metrics. This week I’m going to discuss Product metrics. And while the title of this post sounds somewhat repetitive, as seen in previous articles Product metrics are simply one category that need to be tracked as part of a holistic view of “the product”.
BTW, if you haven’t read the first article in this series, you can read it here: A Model and Metrics for Tracking Product Success.
As mentioned in that article, when thinking about metrics, most people think about the Product first, and for obvious reasons. It’s the focus of the business, and is the most visible and tangible outcome of everyone’s efforts.
And when thinking about Product metrics, the mind most likely turns to the common metrics related to bugs, escalations, etc. All of these are useful, but not as primary measures for product success. Product success needs to be viewed from a higher perspective.
I look at the following list as key areas when tracking product success:
- Strategy
- Roadmap
- Capabilities
- Gaps
- Quality
Keeping abreast of these categories will provide a clear picture of the state of the product over time.
Strategy
Strategy can be a nebulous topic. And while defining product strategy is beyond the scope of this post, there are questions that can be asked to help identify whether the strategy is a good one.
Here are a few of them.
- Is the overall strategy to be market leader or fast follower or something else?
- Is there even a clear product strategy at all?
- Is it to sell into an existing customer base, or to acquire new customers?
- Is the product targeting a specific niche or problem space?
- Have changing market conditions (e.g. new entrants in the market, price pressures, regulatory changes, market awareness etc.) impacted the effectiveness of the strategy?
- What threats (if any) are known that may impact the strategy?
There are likely many other questions that should be asked, but the overall assessment of them will give an measure of how relevant and potentially effective the strategy is.
Roadmap
Ah yes, everyone’s favourite topic. The roadmap ties directly into the product strategy. In short the roadmap is an articulation of that strategy. A well defined strategy enables a clear roadmap. And with a poorly defined strategy, the rule “garbage in, garbage out” takes effect. Regardless, it’s important to think through the roadmap, using the overall strategy as a guide, and assess how well that plan fit in with overall business goals and directions.
Capabilities
This is an honest assessment of what the product can do. Where are the strengths and weaknesses, and where the risks lie in those capabilities. Are there functional areas where competitors consistently win over your product or that customers/prospects say are missing?
Gaps
Gaps are called out explicitly because these are areas that need specific additional investment. And the question really is how much investment, in what time frame, and what is the resultant business impact of making those changes? One word of caution when thinking about product gaps. It’s very easy to fixate on these above other product aspects, such as product strengths. How often have we all heard someone say, “I must have this feature.” or “Without this new feature, we won’t be able to compete.” In how many cases are those statements 100% true?
Monitor and manage product gaps, but view them as one area of focus that doesn’t disproportionately outweigh all others.
Quality
This is something that most product managers are familiar with, having sat in on bug triage sessions, been involved in customer case escalations, or simply hearing about quality issues first hand in customer conversation. Nothing irritates customers more than poor quality products, particularly ones they’ve paid for. 🙂 And they won’t be shy about telling you about those problems.
This is one category where a deep analytic assessment is possible, and that should factor into how quality is measured. i.e. consider measures such as new customer reported bugs, bugs addressed, severity of bugs reported, # of customers reporting bugs etc. All of these can be looked at over time to see if the problems are increasing or decreasing. Based on this assessment, additional investigation into specific problem areas can be conducted.
Putting it all together
As I did for other posts in this series, the following is an example of how these categories can be rendered for easy interpretation.
Other categories
Depending on the product, market and strategy, there may be other product categories to consider, so look at your product and objectives and decide what is important to measure and track.
Overall, the goal of these metrics should be to provide a holistic, yet easy to understand view of important product and business areas, with potential problems or shortcomings clearly visible for discussion and further analysis.
Saeed
Tweet this: @saeedwkhan Product related metrics for Product success http://wp.me/pXBON-2uj #prodmgmt #innovation
| Strategy |
| Roadmap |
| Capabilities |
| Gaps |
| Quality |
| Documentation |