One of the common questions I am asked when discussing our win loss analysis offerings with companies is, “How many deals do I need to analyze before I can start using the data?” In the past, my response has always been that you will get value from the very first win, loss, or no decision that you analyze. However, because statistics teaches us about minimum confidence levels, confidence intervals, and margins of error, people often struggle with the idea that they can start leveraging the data from their win loss program from the very first data point.

Get away from the “survey” mindset

By just listening to the terminology that many executives use, it is obvious that many initially base their vision of a win loss program on their prior knowledge of market research or political polls. For instance, the term “survey” is often used to describe what is actually an in-depth “interview”. Unfortunately, this market research-based mindset often diminishes the usefulness of a win loss program and gives the false impression that the program will need hundreds (or even thousands) of data points to be accurate or useful.

Certainly, if you were to use a win loss program to try to determine the general buying habits of your buyers (or the populace as a whole), you would want to interview a large number of random customers in order to achieve a representative sample and a high level of statistical confidence. However, working with hundreds of companies over the past ten years has shown that the most successful win loss analysis programs do not try to cast such a wide net. Instead, they act more like a laser pointer that illuminates very specific information about very specific issues and questions. In other words, to get the most out of your win loss program, it should not be seen as a survey tool (which simply gathers information), but rather a decision-making tool and an early warning tool.

If you think that you can’t get value from your win loss program, or if you believe you can’t begin making decisions based on your win loss data until you achieve a statistically significant number of data points, you risk missing as much as 75 percent of the potential ROI you could achieve with your win loss analysis program. The key here is in understanding how and when to use the intelligence you are gathering through your win loss analysis.

The decisions executives and managers make

To determine how many interviews are needed for a win loss program to be effective in helping you with your decisions, it is important to understand the kinds of decisions you make in your organization and how win loss analysis can help you make better decisions. Professor Robert A. Harris, in his VirtualSalt blog, outlines three major types of decisions that individuals and organizations make: strategic, tactical, and operational.

Strategic decisions are those that determine the overall direction of an enterprise, product, or department over the long term. They also tend to direct and control an organization’s resources. Strategic decisions involve a high degree of uncertainty due to the scope and time frames usually attached to these types of decisions. Real-world examples of win loss analysis data helping clients make better strategic decisions include:

1. Analyzing competitive losses indicates that buyers want a vendor with strong off-shore capabilities in two specific countries
2. Competitive losses point to buyers having issues with the key contractual terms the client requires of new customers
3. Buyers in both wins and losses describe a specific set of needs that aren’t currently being addressed in the market and warrant the development of a new product

Tactical decisions are used to help support or achieve strategic decisions. In other words, tactical decisions are made to help bring about the results that are envisioned in the organization’s overall strategy. As such, they tend to be more immediate than strategic decisions, and can be made at both the managerial and executive level, whereas strategic decisions are almost exclusively determined by executives. Real-world examples of win loss program data that led to clients making tactical decisions include:

1. Competitive wins indicate that a key factor in buyers’ decisions was the presence of superior sales engineer support
2. Buyers in competitive wins talked about specific sales messaging and sales collateral that helped them make their decision, while buyers in lost opportunities did not reference these things
3. Buyers in losses cited over-confidence of their sales representatives and disparaging the competition as key reasons for non-selection

Operational decisions are everyday choices used to support tactical and strategic decisions. They help to ensure that daily operations go smoothly and that the tactical decisions are implemented. As the name indicates, these are usually made by managers, although they can also be made by executives in some cases. Examples of win loss program intelligence helping clients make operational decisions include:

1. A buyer in a lost opportunity indicates that their sales representative did a very poor job in presenting the solution and did not seem to really understand how their product would solve the buyer’s problems
2. A buyer in a won opportunity indicates that they were having trouble with communication and it was having a negative impact on their decision satisfaction
3. A buyer in what the client thought was a lost opportunity indicates that they are still evaluating solutions and that they would like to hear from the client (this situation happens more often than you think)
4. A buyer in a lost opportunity indicates that they didn’t select the client primarily due to a product feature the buyer thought was missing, but the client’s product actually had the feature the buyer wanted

The number of data points needed to support accurate decision-making increases as you move from operational (which is often immediate and one data point can be sufficient) to strategic, which may require a large amount of information to get a clear picture of the entire situation. For instance, deciding which paper supplier to use for your direct mail campaign (an operational decision) would probably not require you to interview clients on their preferences, but if you wanted to change the focus of your company from services to software (a strategic decision), it would be in your best interest to interview a representative number of your current clients to determine how they would react to such a change.

Win loss as a decision-making and decision support tool

From my experience over the last ten years, I have found that win loss analysis provides the most benefit at the tactical and operational level, although, certainly, it does provide tremendous insight at the strategic level as well. It just takes a little more time before you can start to lean on your win loss program when making those long-term strategic decisions.

A few months ago, I asked myself the question, “On average, how many potential operational decisions are presented to our clients in a typical win, loss, or no decision review?” In other words, how many opportunities for potential change (or no change) will a client be able to identify in reading a profile of a won or lost deal? I began a subjective, ad-hoc analysis to figure this out and help our clients understand the potential value of each completed opportunity analysis. Based on my analysis to date, for each win, loss, or no decision interview performed, you can expect to identify an average of ten operational or tactical decisions that can be acted upon.

When you consider that each analysis is going to help you make an average of ten operational or tactical decisions, it becomes clear very quickly that you can realize great value from the very first interview of your win loss program. Just how much value does each of these potential decisions represent? Let’s look at an example: one of the most common operational decisions identified in win loss reviews are sales weaknesses for an individual sales representative that can be immediately addressed through focused training and/or coaching. If this training can help that sales representative avoid making the same mistakes in future opportunities, their individual win rate will improve. The impact of this one decision to help a sales representative with an area of weakness identified by a buyer could be in the millions of dollars, depending on the size of your sales deals.

If done correctly, every win loss interview performed will net key, actionable intelligence in such areas as:

• What business needs are your buyers trying to solve?
• What could you do to better serve your buyers’ needs?
• What is the character and reputation of your company in the eyes of your buyers?
• How well do your sales teams communicate your value proposition?
• What are the strengths and weaknesses of your products?
• What are the strengths and weaknesses of your competitors’ products?
• How are your costs perceived in relation to those of the competition?

Last year, when I was reviewing some interviews we had done with our own buyers, I saw an issue emerge that definitely led to a better tactical decision. One of our respondents commented that their communication with Primary Intelligence was almost entirely with the salesperson and that they did not have sufficient contact with the Account Consultant who would be handling their program.  Since one of our strategic goals is to be a trusted consultant to our clients rather than just a provider of data, I realized that this was a significant issue. After getting the same feedback in two other interviews, I determined that this was not an isolated case and made the tactical decision to involve our Account Consultants and other subject matter experts early in the sales process. The results have been good, with our clients coming out of the sales process more informed and more committed to their win loss programs than we have seen in the past. This tactical decision was made based on three data points, but would have been entirely missed if we had not been performing win loss reviews on our own opportunities.

So, how many data points do you need?

The case I just described did not require many interviews or data points at all for the win loss analysis to have a positive impact—in fact, it only took one to begin the decision process. This example illustrates what we commonly see working with hundreds of clients. You can and will receive value from your win loss program from the very first opportunity that you analyze. And as you add more data points, you will receive further validation and discover important, new insights.

The point is this: since win loss analysis is a dynamic process, it can and should focus on those issues that are most important to you and your buyers. This means that, unlike a static survey, fewer data points are needed in a win loss analysis to reach the insights you need to make better tactical and operational decisions that will help you achieve your current strategic goals and overall success.

It would be a serious error to believe that you need to wait until you have a “sufficient number” of data points before you can realize value in your win loss program. Don’t make this mistake; begin analyzing and using your win loss data from your very first buyer interview and you will begin seeing returns much more quickly.

About the Author: Ken Allred, Founder and CEO of Primary Intelligence, is a thought leader in SaaS-based sales intelligence, analytics and sales enablement solutions. He is committed to the optimization of sales, marketing and product management teams through the implementation of advanced Sales 2.0 intelligence solutions.

Pragmatic Marketings Jim Foxworthy asked me recently, Is our objective in Win/Loss to learn what to build next, or to learn how to sell more of what we have already built?

We both agreed that win loss programs should be addressing both types of opportunities. However, more often than not, I see win loss programs focused more on what to build next rather than how to sell more of what they already have.

Jim went on to ask, Sales organizations obsess about their selling process (as they should), but win loss analysis keeps a team current to any changes in the BUYING process. And if you know the Buyer Persona and the steps they take to buy, wouldnt you want to mirror that to how you sell?

In recent product management discussions on Twitter, a few product managers commented that personas are usually an ineffective waste of time. This generated some interesting comments and in the discussion Gunjan commented that personas are just assumptions and hence dangerous unless real buyers and users help you prepare them and verify them.

Keep in mind that Buyer Personas are FOR sales primarily. Yes, they can be great tools in helping us make product-related decisions, but there is a tremendous need for accurate buyer personas in the sales process. Consider the following research observations from CSO Insights 2009 research:

1. 50.6% of organizations agree that they need improvement in understanding where they should be focusing their selling efforts.
2. Only 9.1% of organizations report that they are exceeding expectations when it comes to understanding which accounts to focus selling efforts on.
3. More than 40% (41.2%) of sales representatives are not making quota.
4. 61.5% of sales revenue is generated by top 20% of sales reps.
5. 27% of organizations report that analyzing their customers buying process was a top sales effectiveness initiative.
6. 20% of organizations report that revising their sales tools to match buyer needs was a top sales effectiveness initiative.

We can infer from the data above that there is a staggering need for accurate buyer personas based on real-world win loss data. In fact, we recently helped a customer uncover an astonishing revelation about their actual buyers (they have agreed to let me share this anonymously). They indicated that they targeted two key buyers in accounts they were selling to and that these were their primary buyers:

1. C-level executives
2. Sales leaders

With this information in hand, we analyzed their most recent 50 competitive wins. What we found shocked them, but uncovered a considerable opportunity to improve their sales effectiveness through the creation and implementation of accurate buyer personas.

When we analyzed their deals, we found that only 40% of their confirmed buyers fell in the two categories of buyer personas that they had previously identified (C-level and Sales Execs). The other 60% of their buyers fell into two other key buyer roles, with one of those roles making up nearly 40% of all of their actual buyers (as much as the other two roles they had identified as their primary buyers!).

This is a lot like shooting for the corner pocket in pool, only to have your ball bounce around the table and unexpectedly go in the side-pocket. Youre pleased with the result, but the result occurred more from luck than skill. The opportunity for this customer was hugethey could create buyer personas for the newly identified buyers and train the sales force on how to effectively target these two additional buyer types.

This is just one example of how your win loss analysis program can help you build and/or validate conclusive buyer personas. There are numerous other key areas where you can leverage your win loss program to create compelling buyer personas that will not only help you with product-related decisions, but will resonate with your sales force and improve their sales effectiveness.

There are twelve key buyer persona elements that will help a sales person target the right people, align your product’s benefits with the buyer’s needs and help them do these things using your buyer’s language. Each of these elements can be found in your win loss data. These elements are:

1. Buyers key responsibilities
2. Buyers demographics (years of experience, typical team size, who they report to, etc.)
3. Buyers organizational demographics (employees, revenue, industry, growth rates, etc.)
4. Buyers description of their problems and needs (the problems you can solve)
5. Your buyers buying process, in their words
6. Understanding the time it takes your buyers to make decisions (your sales cycle)
7. Buyers perceptions and expectations around the price of your solution
8. What your buyers value most about your solution and why
9. What your buyers identify as the weaknesses of your solution
10. What your buyers see as missing from your solution
11. Sales best practices (from the buyers perspective)
12. Any additional potential buyer personas

The real power in this best practice is that it will improve the probability that you can align all of your resources (product development, management, marketing and sales) to focus on the right buyers, with the right problems, in the right accounts.

So how do these things help a sales representative? The best sales people in an organization are going to have a pretty solid understanding of this information already, but the other 80% of the sales force can get a tremendous amount of value from buyer personas built with your win loss data. A well written and researched buyer persona can undoubtedly help a sales person in both their prospecting efforts and with the deals theyre actively pursuing. Specifically, this information can:

1. Help the sales rep target the right peoplethe appropriate buyers of your solution
2. Improve the sales rep’s ability to uncover buyer problems and needs
3. Boost the sales rep’s ability to speak the language of the buyer more effectively
4. Improve the sales rep’s understanding of how your product helps these buyers
5. Help the sales rep identify other individuals they should be selling to
6. Enhance the sales rep’s knowledge of your buyers’ buying processes (more on this in a future blog post)
7. Increase the effectiveness of sales strategies by refining them to align with what your buyers value most

Buyer personas that are based on data from your win loss analysis program will have much more credibility with your sales force (and the rest of your organization). Make sure that you reference the deals that you used to build the personas.

So how does all of this lead to more sales? If you can help 60% to 80% of your sales force better understand your confirmed buyers, whats most important to your buyers, and how they buy, could having this information help an under-performing sales representative win a deal they would have lost this year? Could it help them win two more deals? Run the numbers on the impact you could have on sales if your win loss analysis-based buyer personas helped sales representatives struggling with their quota close just one additional deal this year. It is a pretty powerful ROI story.

So what do you need to do to get started?

In order to build a good foundation for accurate buyer personas, we recommend, at a minimum, talking to buyers of the last 20 deals you won or lost. Keep in mind that it is important that you try to get an equal weighting of the deals where you won and where you lost. In preparing your win loss interview instrument, be sure that you incorporate the 12 key buyer persona elements Ive identified above.

If you would like to see an example of a buyer persona written using our win loss analysis data, drop me an e-mail.

Defining and validating buyer personas through win loss analysis is a convincing and important best practice that every organization should be doing.

Is your organization following this best practice?

About the Author: Ken Allred, Founder and CEO of Primary Intelligence, is a thought leader in SaaS-based sales intelligence, analytics and sales enablement solutions. He is committed to the optimization of sales, marketing and product management teams through the implementation of advanced Sales 2.0 intelligence solutions.

Now everyone wants to know why you lost.

Your prospect just selected the competition, and youve been tasked with calling the decision makers to find out exactly what happened and learn everything you can about why they chose the competition.

You spend the next three weeks leaving messages, but they dont call you back, or you catch them live on the phone, but they don’t have time to answer your questions. Now what?

One of the constant challenges that we face in the execution of win loss analysis is maximizing response rates or participation rates. That challenge is even greater in this type of market research because sample is limited and you only have a finite number of competitive wins and losses you can study. If we aren’t able to get them to participate, it is a missed learning opportunity that we cant necessarily duplicate.

The example above is probably one youre familiar with if you are involved with your companys win loss program. The question becomes, How can we increase our response rates?

At Primary Intelligence, we spend a lot of time and resources addressing this very question. There are quite a few things we do to increase participation rates for our customers, but one of the most important practices we have identified is to create and utilize what we call introduction notes. These are notes that introduce why you want to talk to the decision maker and what you are hoping to accomplish. While the content of this note is very important to improving response rates, who this note ultimately comes from can be even more important than the words you use . There are three potential options when it comes to deciding who the introduction note will come from:

1. From a key executive (higher the better, with CEO being best)
2. From the sales representative responsible for the deal
3. From you

To successfully implement this practice, you will need to choose which of these three options will work best for your situation and then create two introduction note templatesone for your wins and one for your losses. If you analyze your no decision deals, you will want a different introduction note for that as well.

The introduction note really helps remove the surprise attack feeling that decision makers can get when you are cold calling them to do a debrief of their purchase decision. Cold calling a decision maker is a lot like the solicitor that knocks on your dooryou open the door expecting someone you know and instead get a complete stranger that wants to wash your windows or aerate your lawn. Surprising a decision maker is a sure way to kill response rates for your win loss analysis program.

There are five key elements to include in the introduction note that you will send on behalf of an executive, the sales representative, or yourself:

1. Introduce the executive, the sales rep or yourself at the beginning of the note
2. Introduce the purpose of the e-mailto understand how you can improve
3. Ask them to be candid and forthcomingwe learn the most from our mistakes
4. Identify the amount of time youll need for the interviewbe specific and honest here
5. Let them know when you plan on calling to schedule the interview at their convenience

Even though there are a lot of elements you need to include, it is important that you keep your note brief and to the point. For these notes, straightforward information and a simple request will be your most persuasive argument.

Introduction Note from Key Executive

This is your best option for improving participation rates. You can send the e-mail on behalf of the executive, but you will want the executive to understand what youre doing and have him or her sign off on the processas they could get replies from decision makers and they will need to be in a position to respond to those replies. C-level executive sponsorship will do wonders for increasing response rates, with the CEO being the best possible option.

This approach is easiest if you have a direct executive sponsor. You will be sending the e-mail on behalf of the executive you ultimately get to sponsor your efforts. This way, you have complete control over the process and your efforts wont be held up by the executives busy schedule.

When you have an executive sponsor, you will want to emphasize to respondents that they can reach out to the executive if they have any questions or concerns. Make sure your executive is prepared for this situation, as it can really have a positive impact on decision makers when they see your companys commitment to them and improving your solution to meet their needs.

Introduction Note from Responsible Sales Representative

If you cant get an executive sponsor, the next best solution is to involve sales in the win loss analysis process. An important distinction here is that the sales rep responsible for the deal should not be doing the interview with decision makers. Decision makers will filter out information they perceive to be negative if the sales representative is the one doing the interview with them. See my post on sales-derived versus decision maker-derived win loss analysis for more on this topic.

In order to get sales support, you will likely have to overcome some natural anxiety that the sales representative will have with you contacting the people they were selling to. You will want to reinforce that this process is to identify areas for improvementto help them sell moreand not a witch hunt. Dont be too hard on your sales repsimagine how you would feel if someone said they were going to come in and analyze how you were doing your job. Its natural for them to feel some discomfort, but once they see the results of your efforts and how you intend to leverage those results, this discomfort and anxiety should fade away.

You will want to send the e-mail on behalf of the sales representative so that you can control the process, although we have quite a few customers that have their sales representatives send the introduction notes directly. A sales representative introduction note differs in a few ways from an executive introduction note. You will want to add the following elements to your sales introduction notes:

1. Reinforce that you are not trying to re-engage the sales process (applies to losses and no decisions)
2. Make the request for a debrief a personal request on behalf of the sales repleverage the relationship the sales person has with them

Introduction Note From You

If you are utilizing this method it means that you werent able to get executive sponsorship or sales support in your efforts to review why you are winning and losing. Dont worry, you can overcome this lack of sponsorship and sales support by pressing forward and gathering actionable data that you can share with both executives and sales. Utilize the data to accomplish your objectives, but dont be selfish with it. Share the data with everyone you think could benefit from it, including the CEO (trust me, these guys are ALWAYS interested in why you are winning and losing), product management, product marketing, sales support, sales management, marketing, and sales representatives. In our experience, it only takes sharing a couple of reviewed opportunities (especially losses) before both executives and sales will recognize the value of the intelligence and get on board with the program. However, this cant happen if you arent sharing the dataIve seen too many cases where this valuable intelligence isnt shared across the organization.

The most important thing to remember about introduction notes that are coming from you is that you are letting the decision maker know why you want to talk to them and removing that surprise element. Practicing this will improve your response rates, but your real goal is to upgrade your program to the executive-level introduction notes or the sales representative introduction notes.

Sample Introduction Note Templates

Over the years, Primary Intelligence has created many of the three types of introduction notes I described above. We have spent a lot of time and effort in creating notes that will elicit the kind of response we need to be successful. If you would like a template for one of these introduction note types, send me an e-mail with the template you are interested in and Ill send you a sample of what were currently using to help move you along with this best practice.

About the Author: Ken Allred, Founder and CEO of Primary Intelligence, is a thought leader in SaaS-based sales intelligence, analytics and sales enablement solutions. He is committed to the optimization of sales, marketing and product management teams through the implementation of advanced Sales 2.0 intelligence solutions.

New product failure rates are high. Studies indicate that we can see new product failure rates of nearly 90 percent. A study done in the food industry showed that the failure rate for new products was between 70 and 80 percent, but that the U.S. Top 20 food companies were enjoying a success rate of 76 percent for their new product introductions, while the other 20,000 food companies had only an 11.6 percent success rate for their new products. The biggest difference between the top 20 and the bottom 20,000? The lack of market research.

Studies also show that a third or more of a companys revenue is likely derived from products that did not exist five years ago. There is no question that if a company is to grow and survive, they have to be able to create and launch new products, or at the very least enhance and upgrade their existing products. Take a look at Motorola, a company that led the cell phone industry. At its peak in 2000, Motorola had a market cap that was 8 times larger than their market cap today. Motorola has been struggling for the last couple of years; in fact, their last new cell phone product launch was in 2004. The market has changed tremendously over the last five years, and given the current state of the company, its pretty safe to say that they havent been listening to their markets.

I just realized I used the word product nine times in the preceding two paragraphs to describe the landscape that is product development, management, and marketing. The truth is, it isnt about products. People dont buy products; they buy solutions to their problems. In The Business Battlecard, author Paul ODea quotes Harvard Marketing Professor Theodore Levitt to illustrate the importance of understanding your customers pain:

People dont want to buy a quarter-inch drill. They want a quarter-inch hole.

If we want to create value for our customers, ensure our products success, and secure our companys future, we have to understand our customers needs and problems better than our competitors. There is really only one way to do thatwe have to talk to our market, and more specifically, the decision makers in our target markets.

Based on the new product failure rates that I quoted earlier, we can assume that most product development and management is done using activities performed inside the companyi.e. founders and executives drive product development based on their experience and gut, while occasionally checking in with customers. This is typically the place where everyone has to start when we are creating a new product, but at some point (the earlier the better), the process needs to change from an internally-driven development process to a customer-driven, external process.

In order to create a customer-driven product development process, we must be consistently listening to our market to identify and validate the key customer problems that our solution will solve and measure how well our solution is solving those problems. Pragmatic Marketing defines three groups within our target markets that we need to be actively listening to:

1. Existing Customers: this group includes customers that recognized they had a problem and purchased your solution to solve that problem
2. Evaluators: this group includes both your customers (competitive wins) and those customers that have selected your competition (competitive losses)
3. Potentials: the group of potential customers that havent purchased your type of product, but have tried to solve the same problems through internal efforts

Over the last decade, we here at Primary Intelligence have spent countless hours figuring out how to best listen to the second group identified by Pragmatic Marketingthe Evaluators. Based on that experience, we have found the following seven-step process is an effective methodology for improving your understanding of your markets problems:

1. Make sure you are asking questions that will reveal the problems the customer is trying to solve.
2. Before you begin gathering and evaluating our win loss data, write down your target markets three to five most important or pressing problems.
3. Examine demographic information for the recent wins and losses you are going to interview in order to make sure they are in your target markets (you dont want to identify market problems for markets that you dont want to be in).
4. Perform the win loss analysis and gather the data.
5. Build a repository of the problems identified and validated.
6. Analyze the data to identify problems you may have missed. You also want to validate that the problems you wrote down are real for your markets and not problems that you hope they have.
7. Remember that identifying market problems and validating those problems is a program and not a project. You want to build this process into your weekly/monthly activities to make sure that you are able to stay on top of your markets

STEP ONE: Ask the Right Questions

Make sure that you have your questions prepared before you perform the interview with the decision maker. It will be tempting to just pick up the phone and wing it, but being prepared with the questions you want to get answered before-hand will dramatically improve the probability that you will accomplish your objectives.

Based on our experience, we have found that you need to ask the following questions to both identify and validate market problems:

1. What were your business needs that prompted this evaluation?
2. What were the primary reasons you selected the winning vendor over the other vendors you evaluated?
3. What were the primary reasons you did not select the other vendors you evaluated?
4. What could the vendors you didnt select have done differently to win your business?
5. What could the winning vendor have done differently to better meet your business needs?
6. What did you value most about the vendors solutions you evaluated?
7. What were the biggest solution weaknesses you identified based on your business requirements?
8. What important features were missing from the solution?

There are also several quantitative questions that will help you validate market problems and quantify how well your solution is solving those problems, but Ill need to save that for a follow-up discussion to try to keep the length of this post down.

The first question (what were your business needs that prompted this evaluation?) is your money question when it comes to identifying and understanding your target markets problems. In our experience, the other questions will also identify and validate your customers problems. They wont do it consistently, but they will regularly give you insight into whats important to your customers and why. All of this translates into how well youre solving your customers problems and helps you along the path of becoming stronger experts on your target markets.

STEP TWO: Write Down Your Target Markets Problems

An important part of this ongoing process is validating what you currently think about our customers problems as well as qualifying the impact those problems are having on our customershow important is it to our customers to solve these problems?

An interesting exercise that you can do in conjunction with this is to send an e-mail out to others in your organization (sales, operations, executives, etc.) and ask them what they believe the top three problems are that you solve for your customers, and ask them to rank them in order of importance to the customer.

In preparation for this article, I reached out to our sales, operations, and executive personnel and asked them to identify the top three problems we solve for our customers. I asked them to rank them from most important problem to least important problem. I then took the responses and weighted them to come up with the following list of problems our organization believes we solve for our customers:

Top Three Problems We Solve

According to Us

You will notice that the list above isnt necessarily organized as problem statements. However, organizing things in this way allows us to figure out which category of problems are most important to our target markets. This is perfectly acceptable, as long as you have a deep understanding of the problems your customers are facing and youre able to articulate them using the same words your customers use. The trap that you have to be careful of here is taking a short-cut and falling back onto your benefit statements instead of really dissecting the problems as articulated by decision makers in your target markets.

For example, when one of our decision makers was asked what business problem they were trying to solve when they selected our solution, he responded, We need to gain specific competitive insight on [One of their competitors]. They have been beating us with greater frequency and we need to figure out how to counter this. We can see that this is clearly a Better Competitive Intelligence category problem. In contrast, the following response really belongs in the Better understand target market needs problem category:

We need to get to a more refined understanding of our markets so we can figure out why there is a gap between what we think customers should be spending and what they actually are–we need to find out why this delta exists and use this information to make changes to our go-to market strategies, our product plans and perhaps even our business model.

According to this internally-derived list, the top three problems that I should be looking to validate, in addition to trying to identify new problems, are Competitive Intelligence-related problems, helping customers understand why they win and lose, and helping customers understand the strengths, weaknesses, opportunities, and threats in their current sales process (Sales Process (SWOT)).

STEP THREE: Select Recent Wins and Losses to Review and Qualify Demographics

You want to examine the key demographic information of your recent wins and losses to make sure that they fit within your target markets. The best sales professionals and managers will tell you that there really are only two reasons you lose a deal (versus the more commonly voiced, but generally incorrect, reasons of price and product):

1. You shouldnt have been in the deal in the first place (your solution wasnt a good fit based on the customers needs)
2. You were outsoldthe competition did a better job selling than you did.

Because walking away from any deal is a very difficult thing to do for a sales representative trying to make quota, you will have losses in your database that are not in your target markets. It is very important that you do your best to remove these from the list of competitive wins and losses you will be using to perform your win loss analysis.

STEP FOUR: Interview Decision Makers

To help with best practices when interviewing decision makers, I went to our team of account consultants, the folks that spend at least half of every day talking to decision makers, and asked them for advice they would give. They provided both some best practice dos and donts that will help you improve your interviews and response rates:

Do these things:

1. Pre-notify the responsible sales representative that you will be doing a win loss review on the deal.
2. Let the sales representative know that you will share the results of the review with them so that they can learn from the effort as well.
3. Ask the sales representative to notify the decision maker that you will be calling to do a debrief on how they made their purchase decisionthis will improve response rates significantly. Its important that you craft an e-mail message for the sales representative to send to the decision maker that emphasizes the following:
   1. You are not calling to try to win them back, or re-engage the selling process (this applies to your losses).
   2. You are calling to get their feedback so that you can improve your solution and sales processes.
   3. You are in product management and not sales.
4. Listen. Listen. Then listen some more. There are really only three situations where you should be talking:
   1. Youre asking the interview questions.
   2. Youre asking a follow-up question.
   3. Youre answering a question about one of the questions you just asked.
5. Keep the interview conversationaldont sound like a call center survey.
6. Always know their title, correct spelling of their name and gender prior to the call. Only verify information with them (e.g. I have your title as CFO. Is that the most accurate title?)
7. Make sure you have follow-up, or delving, questions prepared beforehand for each of your questions so that you can get actionable feedback
8. Smile while you interviewit comes through in your voice
9. Thank them for providing candid and informative feedback, especially when the feedback might be considered negative for you, at appropriate times in the interviewthis will encourage them to continue to provide candid feedback.

Dont do these things:

1. Dont go into the interview with a predisposed view of what happened in the deal.
2. Dont lead person you are interviewing, or ask leading questions. For instance, dont ask Did you reject the solution because the price was too expensive for you? Instead, ask something like, Why did you not select the solution?
3. Never assume that you know the answer to a question because of their responses to previous questions.
4. Dont interruptthis will be a challenge because they will say something that you can answer, and you will want to jump in and respond to what they just said, but if you do, you will miss out on critical information.
5. Dont frustrate them with an overly long interview; keep your interview as succinct as you can and be mindful of their time constraints.
6. Dont try to solve all of your problems in one interview.
7. Dont add your own opinion or additional commentary to their responses.

STEP FIVE: Build a Repository

While individual win reports or loss reports are very important to understanding your target markets problems and how well your solution addresses those problems, having a database that you build over time will allow you to validate these problems and identify important trends in the market problems that your solution addresses.

When trending or aggregating information from a group of interviews, keep the following in mind:

1. Try to maintain the integrity of your marketsif you are interested in more than one target market, try to separate your interviews into groups that each fit a single target market and do a trend analysis for each group.
2. When trending, having an equal mix of win reports and loss reports will help avoid positive or negative bias in your overall findings. Being able to see how performance differs between wins and losses will also help pinpoint areas of greatest impact.
3. Remember that certain questions work better for determine overall performance (e.g. performance ratings) and some are more individual-based, making them better for identifying a range of best practices (e.g. an open-ended question asking how the vendor could better serve the customers needs).

STEP SIX: Analyze Stated Customer Problems

Once you have our database set up, you can get to the fun partfiguring out what problems are most important to your target markets and how your solution can help them solve those problems. To help illustrate this, I analyzed ten recent opportunities that evaluated Primary Intelligence (6 wins and 4 losses) to identify the market problems that caused them to seek a solution. The results are very interesting, especially when you compare them with our internally-derived list of customer problems that we solve:

Top Three Problems We Solve

According to Our Evaluators

Keep in mind that this is only ten recent opportunities; however, we can immediately see that we missed an important market problem from our internal listevaluators indicated that improving their current win loss program was one of the most important problems that was driving them to look at our solution.

This process allows me to understand the big picture for our target markets, and analyzing the way each customer communicates their problems gives me fantastic insight into areas that we can/should be focusing on to better meet the needs of our customers. It is comments like the following that help me frame the newly identified customer problem of improving a current win loss analysis program:

“We were doing our own loss reviews, and we were finding that we were not getting good, actionable information from them. Then we found out that Primary Intelligence’s services were available and saw that the level of information was much deeper than we were able to get by doing our own loss reviews. We felt doing our own reviews internally wasn’t as deep because it wasnt our area of expertise.”

“We were trying to gain a more robust process. We were strapped for resources in-house and we were looking for a solution that would allow us to be able to touch more deals than we currently touch, and therefore get a bigger wealth of information as to why we are winning and losing. We have an internal process, but we think we are getting rote answers. We had not gone with a third party ever before.”

We can see that while they definitely want to improve their current win loss analysis programs, our customers are framing this problem around the idea of deeper information, actionable information, wealth of information, etc. These, then, are the real problems underlying the stated problem of improving their current win loss programs.

Analyzing the data in this way helps me understand the problems and their impact, but just as important is the fact that Im able to get a good feel for which problems are most important to our target marketsthat is, which problems are the ones our customers are willing to pay to solve.

The importance ranking that we did internally was quite a bit different than what actually resulted from listening to evaluatorsas you can see when we look at the top ten problems side-by-side:

Top Three Problems We Solve

Evaluator Ranking vs. Internal Ranking

STEP SEVEN: Make Market Problem Identification and Validation a Habit

Remember that win loss analysis isnt a project; its a program. Your recent evaluators are one of the best places (if not the best place) you can go to validate the problems you believe you are solving with your solution.

The things that influence and drive your target markets are fluid and always changing. If there is one constant that we have seen analyzing purchase decisions over the last ten years, it is that markets will change. The problems that you are solving today, while important, may not be the most important problems your customers need you to solve tomorrow. If you are not listening, you can bet one of your competitors will be.

About the Author: Ken Allred, Founder and CEO of Primary Intelligence, is a thought leader in SaaS-based sales intelligence, analytics and sales enablement solutions. He is committed to the optimization of sales, marketing and product management teams through the implementation of advanced Sales 2.0 intelligence solutions.

Recently, respected product managers have been discussing the value of win loss reports as they relate to product strategy and technology roadmaps. In an interesting blog post on Product Management Insights, Michael Shrivathsan argues Why Win/Loss Reports Shouldnt Drive Your Product Roadmap. In his article, Shrivathsan outlines why doing win loss reports with your sales team might be unreliable, including the fact that there might be hidden bias in the sales teams perceptions. He cites a humorous tweet from Mike Boudreaux, a fellow Product Management twitterer, to illustrate his point: Typical win/loss analysis from sales force: majority of losses due to product and price. Majority of wins due to relationship.

As I read this article, I could understand Shrivathsans argument, but only if you define and develop your win loss program in a narrow and short-sighted way. By operationalizing your win loss in the way described, or by abandoning win loss analysis altogether, youll be missing out on perhaps the most important intelligence you need to validate your product strategies and roadmaps.

To begin, an important distinction needs to be made concerning the win loss analysis methodology being discussed in Shrivathsans post. From his comments, we can infer that the win loss analysis methodology being utilized is to have the responsible sales representatives identify the reasons they won or lost a deal and report this information back to product management and other interested parties. I define this as a sales-derived win loss analysis methodology.

This is a common conception of win loss analysis. Based on our experience at Primary Intelligence, we know that approximately 1/3 of companies indicate that they perform win loss analysis; however, we have found that only about 50% of those companies that say they have a win loss program implement what we define as a decision maker-derived win loss analysis methodology as opposed to a sales-derived one.

The distinction between these two very different types of win loss programs is very important. In fact, to fully illustrate this point I went to our own win loss program and grabbed a recent real-world example. This is an analysis of a recent competitive win, but the principal holds true for losses as well (if there is interest, I can show a real-world loss example at a later date).

Lets compare the results between a sales-derived win review and a decision maker-derived win review.

Selection Reasons

When our sales representative was asked why we won the opportunity, his response was:

The reason we won the deal is because I was tenacious and kept working on the deal and building relationships until I got in front of the right person. I then built my relationship with that person so that we had a great relationship. It was good that we had technology, but the primary reason they selected us was because of the relationship I was able to build with the decision maker.

Does this sound familiar? This response sounds similar to the kind that Shrivathsan laments in his blog post. However, lets compare our sales representatives response to the decision makers response on why they selected Primary Intelligence:

There were a number of reasons. One is their extensive experience in this business. [PIs sales rep] and [PIs account consultant]s personalities were a compelling factor, as were the quality of the work and the technology. We really liked the dashboard. Were just now starting to tap into the power of the technology. Primary Intelligence also gave us the ability of integrating the win loss data into Salesforce.com, our CRM system. The others could not do that. Those would be the primary selection reasons.

We can see from the decision makers comments that our sales reps relationship with the decision maker (along with our account consultant) absolutely had an impact on the decision. However, if thats all the information we had, we would be missing valuable intelligence on how this customer actually arrived at their purchase decision.

More importantly, our product management team could be led to believe that the integration with our customers SFA tools we have been investing in may not be a high priority for our customers and we could begin to question our product strategy.

Competitor Weaknesses

An important element of every win loss program is gaining a better understanding of your competitors products. Lets take a look at the competitive intelligence gaps we had in the different win loss analysis methodologies.

Our sales representative was asked why our primary competitor lost the deal. His response was:

They werent pleased with the depth of the information [our competitor] showed in their samples. The decision maker was really looking for a strategic partner and they didnt feel they could get that from [our competitor].

And the decision makers response to why they didnt select our competitor:

Primarily it was the quality of their work. I found the quality of the work was not very good and the questions were not followed up on. The deliverable was fairly weak and looked very unprofessional.

Our sales representative did a pretty good job describing the situation, but the decision makers response is a lot clearer when it comes to trying to understand our competitors weaknesses. The decision maker identifies three distinct weaknesses, while our sales rep identified only two weaknessesone that was confirmed by the decision maker and one that wasnt.

Valued Solution Features

Now lets dig a little deeper and focus on the solution features to see if we can identify additional gaps between sales-derived and decision maker-derived win loss programs. We asked our sales representative what features of the evaluated solutions were valued the most by the decision maker:

Quality of interviewing, because we do a much better job than [PIs competitor] and really uncover the things that affect the outcome of purchase decisions.

Quality of interviewing is definitely an important feature of our solution, but was it really the feature that the decision maker valued most? In our win loss interview, we asked the customer to rate the quality of several product criteria for both PIs solution and the solutions offered by the competition:

We can see here that while our sales rep was partially correct in that we did perform well in the quality of interviewing decision criterion, the decision maker indicated that the quality of deliverable and tools and dashboards were what they valued most. The customer commented:

Primary Intelligences deliverable is very balanced between quantitative and qualitative information. The executive summary thats produced saves me a lot of time and effort and I get a lot of value from the analytics we get like the competitive advantage scores and predictive analytics. And the value of delivering everything that is produced through the dashboard is really important.

If our current product strategy is to invest in specific technology features and we were to see several sales-derived win reviews like this one, we might begin to question our current plans. Seeing enough reviews like this might even cause us to halt development and redirect those resources to other projects. You can see how it would be easy to make some very costly mistakes if you are ONLY reviewing sales-derived win loss data.

Conclusion

The following table clearly illustrates the large differences between the two methodologies we have been discussing:

If you, or your organization, currently defines win loss analysis as a debrief from the responsible sales representative, I hope these examples can help shine some light on the big, risky gaps that are inherent in performing post-decision analysis via the sales representative.

To be perfectly fair, Shrivathsan does make a brief mention of including decision makers in win loss analysis, saying, theyre often much more open & honest to a product manager with whom they have no relationship, than a sales rep who worked with them for a period of time. As he alludes to, and as we have found in our own experience, these same principals weve been discussing will apply if you are having your sales representatives interview decision makers, so its important to have a third party of some sort perform the analysis (Ill be posting on this subject in more depth at a later point).

I have to agree with Shrivathsans assessment that you shouldnt allow sales rep-derived win loss reports to drive or affect your product roadmaps?but only if you are performing sales-derived win loss analysis. Even in our organization, where we specialize in helping companies implement win loss programs, if our product managers relied on win loss reports from sales reps alone to drive product strategy, we could quickly get ourselves into trouble. Fortunately, we have learned to incorporate a superior source of informationcustomers and prospectsthat provide us with data we need to make a roadmap we can trust and that will ultimately allow us to serve our customers needs more effectively.

In my next post I will write about how you can get a tremendous amount of value from sales-derived win loss reports and how to appropriately use them.

Note: If youve never had the opportunity to review a decision maker-derived win loss report I would like to change that now and extend an offer to do a couple win loss reviews for freeno expectation or obligation on your part. Contact me if youre interested in taking me up on this offer.

About the Author: Ken Allred, Founder and CEO of Primary Intelligence, is a thought leader in SaaS-based sales intelligence, analytics and sales enablement solutions. He is committed to the optimization of sales, marketing and product management teams through the implementation of advanced Sales 2.0 intelligence solutions.

A case in point: a few months ago, after speaking with our sales team, I had been thinking about our sales process. At the time, I thought, I need to make sure that we get our Account Consultants involved early in the sales process.

Our Account Consultants are the folks that work hand-in-hand with our customers to implement and run their win loss analysis programs. When it comes to win loss analysis, these are the experts that are deep in the trenches, evaluating purchase decisions with our customers and helping them solve their business problems every day. I realized that if we had our Account Consultants involved earlier, we would potentially be doing a better job in helping customers evaluate how our solutions can help them achieve their goals.

As with my other ideas, this change to the sales process went on my list. Unfortunately, that is where it has remained. Then this morning, as I was reviewing a review we did on one of our recent losses to a competitor, I see a response to the question, What were the primary reasons you did not select Primary Intelligence? The decision maker commented:

The only thing I would say about Primary Intelligence was that in the initial part of the sales cycle I was talking with more of a client executive, someone who was more of a relationship manager. I understand that model. I understand why you have it. It’s not a bad model. But with [PIs competitor] I was speaking directly with the partner who was going to actually execute the work. There was no middleman. We were nose to nose on the discussions, and I felt that the sales cycle with [PIs competitor] moved a little more quickly because there was no middle salesman. That was really the one slightly negative thing that I had toward Primary Intelligence. I definitely would have preferred to speak with the person who would be working directly with me on the project during the sales process.

Ugh. Yeah, I had that sick feeling in my stomach. If I had only acted on the idea to improve the sales process earlier I would have positioned us better and given us a much better opportunity at this customers business.

I am pleased to report that I officially checked this idea off my list after a couple of conversations with our Director of Operations and our VP of Sales. They are in the process of formalizing the change to our sales process so that our customers will be working directly with our experts right from the beginning. Our customers will have full and immediate access to both their sales contact and our subject matter expertstheir Account Consultants.

The point here is that even after doing this for the past decade, we still consistently find important, even critical, intelligence on how to better meet the needs of our customers through our own win loss program. If it wasnt for our win loss program, I am fairly certain that this idea to improve our sales process would still be on my list and we would continue to miss opportunities.

About the Author: Ken Allred, Founder and CEO of Primary Intelligence, is a thought leader in SaaS-based sales intelligence, analytics and sales enablement solutions. He is committed to the optimization of sales, marketing and product management teams through the implementation of advanced Sales 2.0 intelligence solutions.

The problem is, how does the inexperienced sales rep learn these valuable lessons? When it comes to understanding competitors, most salespeople learn in two ways: experience in competing against vendors over time and anecdotal comments made by colleagues. This isn’t good news for someone new. They end up losing plenty of potential business as they learn on the fly.

Primary Intelligence created Competitive Navigator to give all salespeople access to the competitive intelligence that would normally take years to obtain on their own. With a central intelligence location, all sales people receive the same training and information that helps them compete against any competitor.

Primary Intelligence does this by interviewing prospects and clients from your previous sales scenarios, regardless of whether they were wins or losses. In doing so, Primary Intelligence receives perceptions of vendors and their products directly from the person who is evaluating and buying them.

Whenever a competitor is encountered, Primary Intelligence asks key information about the prospect’s perceptions of the company, its solutions, and its sales team activities. As more instances of competitor activity are encountered, the picture becomes clearer on the market’s view of the vendor’s products, reputation, and sales practices.

Primary Intelligence is also gathering intelligence about your own company, solutions, and sales practices at the same time. Therefore, it is possible to compare your strengths and weaknesses against those of each competitor. By doing so, Competitive Navigator points out strategic methods you can use to beat any competitor in any sales scenario.

For more information about Competitive Navigator, click here.