by Anders Linsdorf
Every company wants to be a success, and of course, that means having successful products. But what is a successful product?
Different companies will have different goals and it’s critical to measure and track product performance to achieve product success. While there can be many possible measures (metrics), the important ones are called KPIs (Key Performance Indicators), and finding the right KPIs and avoiding so-called vanity metrics is critical.
What’s your “On Base Percentage”
In Michael Lewis’ book Moneyball
Nevertheless, the philosophy went from the premise that the number of bases earned is what wins games; not number of home runs hit, or number of stolen bases or any other metric.
It all came down to one thing: “on base percentage”. How often will the player gets on base? This insight allowed the Oakland A’s to perform WELL above average given their limited budget and exploit some loop holes in the market.
This is a powerful reminder that focusing on one thing can be very powerful. It will eliminate discussions about which of the KPIs is more important in case one goes up and another goes down. It will also ensure that everyone is on the same page and no-one is in doubt about what success looks like. So the first step towards product success is to find your “on base percentage”.
Different types of KPIs
There are two main types of KPIs: objective and subjective. They have different properties and can be used in different circumstances.
Objective KPIs are the best, because they pick out a measure that can be verified regardless of human perception and interpretation. They are things that can be measured like: frequency, volume, amount, duration etc. Most of the classic web analytics and economic parameters fall in this class. A good example of an objective KPI is downloads per day for an app. There is no way you can argue with that. The same goes for units sold per day.
Subjective KPIs measure things that depend on human assessment and interpretation. This does not mean that they are less quantifiable. When you measure customer satisfaction, there are many good frameworks for doing that. It could be Facebook likes, retweets, NetPromoter Score, shares etc. These are all quantifiable measures of a subjective quality, whether that is satisfaction, interest, pride or something else.
The business model determines the KPI
The absolute central KPI will always be monetary, like revenue, margin, or equity and the success of a product will in most cases be measured directly by the amount of value it generates.
Some people may object to money being the central purpose of a product, especially if they are of an idealist anti-capitalist inclination. But since money pays for salaries, electricity, office space, health benefits, taxes etc, the success of the product must somehow be traceable to this.
Even if you are a non-profit charity organisation you need to pay the rent, electricity, salaries etc. Some products are offered for free and are viewed as successes even though they generate little or no revenue. e.g. Quora, Pinterest etc. But the reason they are valuable in the eyes of investors is the promise of revenue, Typically the KPI is number of users. But this figure is usually coverted into revenue by a value those users can potentially generate. (eg. $10 per user per year). In that sense money is once again the measure for product success.
Revenue
The most straight forward measure of product success is the revenue it generates. The reason is that it is the cleanest and most intuitive metric. Nobody can argue with that figure – money in the bank is money in the bank.
Sometimes revenue is more important than anything else, since you can’t argue with liquidity. Examples of these types of products are SaaS products where it is popular to track Monthly Recurring Revenue per product. That is an accepted standard. Traditional software companies typically track revenue from licenses sold.
Transaction-based companies like credit card companies or other types of infrastructure companies who charge by transaction may track the number of transactions or transactions per customer per month etc.
It may occur that the revenue is not directly referable to the product in itself. That is the case for many free products (but not freemium products).
A good example are open source software products. If we take the Cassandra database as an example, it is free and can be downloaded. It is built and maintained by Datastax which received a very large round of funding recently $106 million ($189M in total). You could track Cassandra’s success in its installed base.
Then again, most people need support, since it is after all a complicated product, and guess who is selling SLAs, support and implementation consultancy for Cassandra? Datastax of course, so again the KPI (installed base) is a proxy for revenue.
More interestingly this creates a drive towards making the product more needy of consultancy, so what looks like a successful product to the company is not necessarily the same to the customer in this case.
Revenue is a good around indicator if the business model dictates you earn money on all the units shifted.
Margins
Revenue is not always an indicator of business success as you could be selling products for a lower cost than the production cost. More sales would therefore not lead to success for the company.
Margins are a more precise way of tracking product success because they directly support business success. Margins are typically calculated as profits/revenue. i.e. profitability can be tracked as revenues grow.
It is a good thing to track margins when cost per unit is easy to calculate. That is usually the case for hardware and physical products, where cost per unit is often already calculated.
So, in the manufacturing industry that would be a good measure. The same goes for retail, where the cost per unit is the purchase price, plus transport and storage. Sometimes the margins are just calculated as the purchasing price minus the sales price. That is good rough measure of success.
In other industries, like service industries, it is a bit more difficult. If we take a hotel, the cost per unit is somewhat more difficult to calculate, since heating, cleaning and electricity is not typically calculated per room.
The same goes for professional services, where one hour sold may have the cost of the consultant for that hour, but also all the other hours he or she didn’t bill to any customers and that amount is very variable.
Margins are a good KPI for industries where the Cost per unit is easy to calculate and intuitive to understand.
Equity
In some cases revenue and margins may fall short as measures of success. To measure product success in terms of equity is more prevalent in isolated fields.
In accounting terms, equity is the the amount of liabilities minus the assets. For a product that would mean how much you owe (for buying, producing, building etc.) minus the market value. That doesn’t make much sense in terms of serial produced products like cell phones.
If we take real estate it makes a lot of sense. If the product is a house or an apartment, the ability of investments in this apartment to raise the market value is a very good measure of success.
Something similar is also the case for incubators, where the start up itself is the product. Here you may want to track the investment against the market value in order to see whether the product is successful. I would guess that Y-Combinator, Rocket internet or 500 start ups would track the equity of each individual start up rather than their revenue or margins.
One problem with equity though is that market value may be very hard to assess before you sell the product. That makes it hard to use as a proactive KPI. But usually there are other proxy indicators. In consumer technologies, the standard one is number of users, downloads, visitors etc. because that can be used to calculate predicted revenue and therefore market value.
Equity is a good measure if the product is unique.
Non-monetary measures
There may still be instances where you would measure product success by some other KPI, which is not monetary.
One example is satisfaction. Many products from state departments are offered as a service to citizens. They are still products, but they do not generate any revenue, margins or equity, indeed, they were never meant to. Instead they generate a service, the success of which can be measured by satisfaction in one sense or another.
Satisfaction is somewhat more difficult to measure, because it depends on subjective measures.
A rating scale is a typical solution, but it could also be sentiment analysis from social media or clicks on icons such as likes and smileys. Number of issues registered or complaints related to a product would be another way of tracking satisfaction.
Sharing on social media is similarly usually a good indication of satisfaction, but you can’t tell whether it is shared because of a positive or negative experience.
Some public sector products are however not meant to have high user satisfaction ratings: Jails will not be measured by user satisfaction either, since the users are meant to not have great satisfaction from it.
Non monetary measures are good when the business does not generate it’s operating budget from the product.
Recommendations
Product success is key to the success of the entire enterprise. You should carefully study the business model of the enterprise and pick just one KPI as the measure of success. The KPI should be quantifiable and possible to track continuously. Otherwise you don’t have anything to steer after.
You could and should still track other KPIs if they in some way influence the central KPI. That way you can learn the best ways to optimise your product and see early warnings of problems.
Anders
NOTE: This article was originally published on SensorSix.com. An edited version is reprinted here with permission of the author.
Tweet this: What is a Successful Product? An Overview of Metrics and KPIs to Track Product Success http://wp.me/pXBON-49v #metrics #kpi #prodmgmt
About the Author
Anders works to to turn new innovative ideas into the reality of tomorrow. His company, Sensor Six, makes software to help companies develop new products.