There’s an adage that states, you can’t manage what you can’t measure.
And if that is the case, then you have to wonder how much actual management of products, Product Management departments are doing?
Why? Because when you talk to Product Managers about product metrics, it’s usually a pretty short conversation. 🙁
Unfortunately the metrics a lot of PMs are most closely following relate to bug counts or velocity in an iteration, or simple high level business metrics such as quarterly sales or average deal size (ADS).
Not that these are bad metrics, but they shouldn’t be the ones that dominate focus to the exclusion of a more holistic view of the product.
There is some discussion on this topic happening in the blogging community.
Cindy Alvarez had a post recently entitled 57 Questions About Metrics. There are a lot (57 in fact!) of good questions listed there, with categories such as:
- Acquisition
- Activation
- Retention
- Referral
- Revenue
But that list of questions is heavily skewed towards funnel metrics for SaaS/Web applications. It’s also not clear which of the 57 questions are more important or less important given your situation. But, it’s a good list nonetheless.
Brainmates have a document on their site entitled How do you manage the financial health of your product?
This is also a good document, but again focuses on SaaS applications with metrics like Cost per Subscriber, Monthly Disconnect Rate, Total Operating Costs etc.
It’s important for Product Managers — particularly those who are more focused on the business side of product (i.e. product MANAGERS) — to look at the state of a product more holistically, dare I say, “cross-functionally”, to manage it effectively.
Managing Products Holistically
This topic is something that I’ve thought about quite a bit. I wrote several articles last year on the topic of Product Management Metrics. But I didn’t get into specific metrics in those posts. I’m not sure why exactly, but the fact that I didn’t — even though that was my intent — is likely an indication of the difficulty of nailing this topic. I think I’ve found a good general model that can be applied in a wide variety of cases regardless of product type, deployment type etc.
Before I continue, I’d like to thank Tom Grant, Sue Raisty-Egami, Jim Holland and Alan Armstrong for helping me find some more clarity on this subject and giving me feedback on my ideas.
To manage products holistically, there are 4 categories that Product Management needs to understand. These are:
- Business
- Go-to-Market
- Organization
- Product
Business Objectives
I’ve listed this first, because it is the most important. Product Management has to be focused on business success of the product and a clear picture of the state of the business for any product is critical.This could include many things, but metrics could include Revenue (weekly, monthly, quarterly etc.), Deal Pipeline, Win/Loss numbers, Average Deal Size, Renewal Rates, Pricing etc.
Go-to-Market Objectives
This includes major areas of go-to-market readiness such as proper positioning, competitive analysis, customer references, status (and plan) of awareness activities, collateral etc. These and other GTM activities need to be assessed to ensure objectives are clear and focus is being placed in the right areas. Note that these are not all activities that Product Management (or Product Marketing) will explicitly complete (other marketing teams will likely do much of this work), but Product Management must have a clear picture into the state and effectiveness of these activities.
Organizational Objectives
This category focuses on cross-functional readiness across the organization. While the Go-to-Market objectives cover a lot of Marketing activities, other groups including Sales, Sales Consulting, Technical Support, Professional Services, Customer Education etc. all need to be assessed to ensure they are enabled and have the knowledge and tools they need to do their jobs. Any gaps found in this assessment can be acted upon to ensure those teams get enabled and are supplied with the necessary information or tools as needed.
Product Objectives
I’ve listed this last simply because, it’s the one most people focus on (first!) and I wanted to emphasize the first 3. A great product is DOA if the rest of company cannot market, sell and support it. Product objectives include Product Strategy, Capabilities, Gaps, Quality, Performance, Stability etc. i.e. high level aspects of the product that could impact sales or customer satisfaction.
These 4 categories, and the specific metrics measured in them can provide a very clear picture of both the state of a product and any areas of strength/weakness. Weak areas should be acted up to address the underlying issues causing the weakness.
Objectives change across the Product Lifecycle
There is one other aspect to fold into this model. Depending on the stage of the Product Lifecycle a product is in, there will be a different emphasis on the specifics of the 4 categories of Objectives listed above. For example, Organizational Readiness is absolutely critical when a product is in growth stage, but far less critical towards end-of-life. Or at a more granular level, product stability is less important in the early develop/launch stage (when there are very few customers) but critical once the product enters a growth or maturity stage.
In short, the OBJECTIVES of the stage of the Product Lifecycle a product is in, determine the focus of the objectives when measuring the product.
Back in my post Product Management Metrics (part 3), I talked about the stages of the product lifecycle and how each stage had different objectives. I’ve summarized them in the following table:
If you take the Product Lifecycle stages/objectives and tie them into the 4 categories for a holistic product picture, you get a grid that looks something like this.
This diagram shows some of the differences in Product Management FOCUS for the objectives, across the stages of the lifecycle. i.e. The focus for these categories will NOT be constant across the lifecycle. i.e Organizational Readiness is less important early on, but critical once the product reaches growth stage and beyond.
NOTE: This diagram won’t be 100% the same for all products in all companies, but I’ve created it based on my own experience managing products across the lifecycle stages. But please modify these as you see fit for your company and products.
Now, for each of these boxes, there will be a number of specific metrics to measure. These are really up to you to define. I’ve listed a number of them for each objective, earlier in this article. In some cases, you could use some of the metrics listed in Cindy Alvarez’s article, or in the Brainmates article. There is NO silver bullet here. It completely depends on your company, product strategy and goals.
But, you should be able to come up with a standard set (say 5-7 maximum) of key metrics for each objective (i.e. row) that are meaningful across all stages of the lifecycle. In some stages they won’t be as important (e.g. # of customer references) as in others, but once you come up with those metrics and define a clear way to measure them, they just become part of the overall picture that you look at when measuring product success. Keep in mind that for any given product, you only have to worry about 1 column at a time. i.e. it will only be in one stage of the lifecycle and so that’s all you have to measure.
So, there you have it, a general model to use. What do you think? Make sense? Useful? Yes? No? Have questions? Regardless, I’d like to hear from you.
Saeed
Additional Articles: The follow articles drill down into this topic in more detail.
- Business-oriented metrics for Product Success
- Organization and Go-to-Market Metrics for Product Success
- Product Metrics for Product Success
Tweet this: @saeedwkhan A Model and Metrics for Tracking Product Success http://wp.me/pXBON-2oL #prodmgmt #prodmktg #innovation