AlanProduct ManagementWin/Loss Analysis

Dear Alan: 15 minute interview guide for Win/Loss Analysis?

I recently had a request asking for help to design a win/loss analysis interview. As this topic has general applicability across industries, I thought I’d share some of that exchange with you, dear readers.

Question from PM: “My colleagues and I here are looking for a good but brief 15 minute interview guide for win/loss analysis.”

My response (AA): Why 15 minutes? I generally find it takes 30 minutes to get to the real answers. Also what is the ASP for your product? If you work large deals, I again question the 15 minute limit. Are you planning to do a single interview per sales opportunity? How many stakeholders do you generally have in a sale?

PM: 15 minutes is what we were asked to limit ourselves to by the Sales rep I believe.  I agree with your point that more would be better.  We can try for that.

AA: If all of my earlier assumptions are correct, then a 15 minute interview could actually be worse than no interview at all. It takes most interviewees about 10-15 minutes to warm up to the interviewer and start to open up. I spend the first 15 minutes helping the interviewee get comfortable with me by asking some feel-good questions such as their own professional background, the story leading up to the consideration, and so on. This also allows me to understand the real drivers of the decision as opposed to the final conclusions.  I don’t ask my high-value questions (about final conclusions) until I reach that tipping point of trust and openness with the interviewee. So if you start with your high-value questions right off the bat, you are going to get the defensive answers, not the open answers. Also you will not have any way to question the answers you get because you haven’t studied the underlying drivers.

So in 15 minutes, you will get the same answers that the sales rep already knows. You’d be better off interviewing the sales rep and recognizing that it’s sales’ opinion. If you interview the client for 15 minutes, it becomes the client’s answer when it’s really just the surface answer. That’s why it’s worse to do the interview – it just legitimizes the surface answers and gives them more weight. The sales rep will likely say (to you directly or to themselves), “you see, I knew all that. That’s why I don’t think you needed to do that interview, and that’s why I’m only ever going to give you 15 minutes with *my* customers.

Most of my interviews are 30-60 minutes in length. No sales people allowed on the call. In your situation, I would guess that you need 3 interviews per account (just a guess), and plan on 45 minutes per interview.
Final point: It’s critical that you understand who owns the account. Everyone thinks that sales (and more often, the sales person) owns the account. That’s not true. The company owns the account and provides almost exclusive access to sales because we delegate revenue production to sales. However the company needs to retain the sense that sales is just the steward of the account. When the company needs information like win/loss analysis to remain competitive, this is something the company has every right to perform. The company needs to do it in a way that does not disrupt sales, but neither can the company be prevented from doing its research by overly-protective sales team. It’s a balance, but we need to remember who owns the account.
And if you are not getting access, often only the CEO can fix it. You need to sell the need to the CEO.
Readers, beware: Most of you can probably do a lot with a single interview per account, but 15 minute interviews are worse than no interview at all. 30 minutes is a minimum, and no sales people are allowed to join the call.
Hope that helps.
– Alan
  1. Kim Gusta

    If you want more tips on doing Win/Loss interviews and developing buyer personas (I assume you’ll create personas based off your research), check out Adele Revella’s website http://www.buyerpersona.com

    I learned how to do buyer research from her when she taught for Pragmatic Marketing. In my opinion, she’s the best qualified expert on Win/Loss, buyer research, and personas. She also has a very good e-book on how and why to create personas http://www.buyerpersona.com/persona-marketing-ebook and she has some free persona templates on her website.

  2. Derrek cooper

    I think it goes w/o saying, it’s quality over quantity. In general, I agree with what you are saying, but not sure I look at it the same. “long” extensive interviews are good because you are able to dig a bit deeper and get past the obvious. But I would say having 3 15-20 min conversations is more beneficial to me than 1 60min in many cases. The key is “dialog and conversation”. Know your customers, engag with them often.

    1. Alan Armstrong

      Thanks Derrek, I think we’re both right 🙂 In the specific case I was addressing, these guys were getting a single shot to talk with the lost buyer. So it was not a “customer” interview, but a “loss” interview. I agree with what you say, i.e., that a 15-20 minute conversation is great with someone you already know and have a relationship with. Also if you are asking specific, directed, “closed” questions (as opposed to more open-ended exploratory questions), you don’t really need them to open up much and you can get what you need pretty quickly.

      Things are different with loss interviews, and even win interviews. A lost account will usually give your sales rep a reason why they were not chosen. Often the reasons provided come back to features and price. Let’s assume that the reason provided is actually true (sometimes it’s not). Knowing that you lost a deal because you didn’t have feature X or because your price was too high doesn’t really help you. How many times have you built feature X, then found out you keep losing but for other reasons. The point is that in order to get to the heart of the matter, we need to look below feature X. We need to interview about their business goals, perceptions of the options, how those perceptions were formed, what influenced those perceptions, and then drill into the most important areas. We might find out that feature X was important, but we often find that there are other aspects that were much more influential in losing the deal. These might relate to ease of implementation, the reputation of your support or consulting teams, or other factors not related to feature X.

      We can’t get to those kinds of answers with a 15 minute interview. Again I agree with you that short conversations can be good with customers, when you have a relationship, or when you are looking for a very specific answer to a specific question or small area of questions.

  3. Nina Swienton

    Completely agree with all points. One question: what to do when you’re dealing with a multi-tier distribution channel? Meaning, your company’s VAR’s/Dealers ‘own’ the end-user and are hesitant to give corp. pdt marketing people access to it?

Leave a Reply

Your email address will not be published. Required fields are marked *