Win-loss Analysis FAQ – Everything you need to know

Home /  Blog /

• This guide aims to answer the most common questions we’re asked about win-loss.
• Check out the table of contents below and skip right to the section you need.

Win-Loss Analysis Foundational Information

What is win-loss analysis?

A win-loss analysis helps businesses discover why customers choose their products or not.

The process involves speaking to buyers directly, either by phone interviews or by surveys.

Win-loss analysis data is gathered from a number of places:

• 1-to-1 win-loss interviews over the phone
• Surveys sent to buyers
• CRM data such as closed-lost reasons.

What types of companies use win-loss analysis?

Win-loss is most commonly used by businesses that sell products or services with a moderate to high annual cost (usually $10,000 USD or greater), but have sales volumes in the thousands or below rather than millions.

Once businesses sell millions of products, they are able to test entire new offerings in sample geographies, split test pricing, and conduct scale-based studies.

Companies with smaller volumes of sales and less commoditized products seek win-loss to understand in more depth what their buyers think. The higher value of their products means that there’s a considerable benefit to conducting 1-to-1 buyer research.

Why is win-loss analysis important?

Simply: because it fills a gap that many businesses are suffering from.

Companies tend to have a host of data gauge how well their product sells: accounting data, funnel analytics, and CSAT scores. All of them are quantitative analytics.

Win-loss fills “The Why Gap,” which is the understanding of why customers buy.

Are win-loss reasons the same as closed-lost reasons?

Not really. Closed lost reasons focus only on lost deals. A minority of companies do ask their clients for win reasons.

The real difference lies in where and how the data is sourced from. 

Closed lost (and win) reasons are entered by the sales team after a deal is successfully signed, or declined. They ask the client their reason and enter it into the CRM. It’s such a common feature that most CRM software has them enabled by default.

The reality for salespeople is that sometimes they just don’t know why a deal was lost. Lots of deals ghost them or give vague answers such as “it just wasn’t the right time,” which is a bit like a boyfriend saying “it’s not you it’s me” during a breakup. It’s an attempt at a polite answer.

For that reason, closed lost data come with a number of downsides:

• They are reported by the salesperson – as prone to error and bias.
• Sales reps generally have to choose one. In reality, purchasing decisions are made for a multitude of reasons.
• There are often only 4-8 options.

Where do you gather win-loss data from?

Really, it comes down to the following three. Sure, sometimes there are supplementary data included, but I’ve listed the core ones.

• Interview
• Surveys
• CRM Data

Each has its own pros and cons.

What are the pros and cons of win-loss interviews?

Pros:

• Deep-diving interviews uncover new information and ideas you hadn’t considered.
• Able to probe into the root cause of problems. 
• Conversations yield more information per minute than a survey.

Cons:

• Win-loss interviews are time-consuming to administer and analyze.
• As such, win-loss interviews are expensive (in comparison to surveys).
• A large number of administrative tasks include existing data analysis, contact lists, outreach segmentation, sending and tracking invites, interview design, sending interview incentives, call recording, transcription, and analysis. 
• Qualitative research is time-consuming and difficult to codify and analyze.

What are the pros and cons of win-loss surveys?

Pros:

• Fast to create, very low barrier to create.
• Cheap to send. Thousands of surveys can be sent for hundreds of dollars.
• Countless software options are available.
  • TypeForm is a simple entry-level option.
  • Despite its humble beginnings, SurveyMonkey now boasts enterprise-ready customization options.
  • There are a number of enterprise options that tend to need annual subscriptions.
• Easy analysis: Results are generally quantitative and categorical, which are easier to analyze quickly with basic tools like Excel.

Cons:

• Survey-creator sets the options that a survey taker can choose from, which can risk biasing information.
• Survey takers see “Other” options in surveys and free-text fields as hard to use (especially on mobile), meaning new ideas rarely surface.

What are the pros and cons of win-loss data from the CRM?

Pros:

• Pre-configured in the CRM
• Easy to enter
• Easy analysis. Similar to surveys, entries are generally quantitative and categorical, which are pretty simple to analyze

Cons:

• It’s another field for sales reps to fill out. The more fields and admin they have, the more often they default to easy options instead of asking (missing feature, pricing).
• False answers. Clients or prospects are polite. They rarely will tell a salesperson to their face, “your competitor’s sales team was just more impressive.” They are more likely to say, “it’s just not the right time.” 

The ideal mix is all three.

• CRM data tell you where to win and lose more deals.
• Interviews gather depth but are expensive.
• Surveys are shallow but cheap and scale well.

Everyone has some sort of budget limit.

I recommend interviewing as many people as your time and budget allow. Uncover as many new ideas and concepts as you can, then turn to surveys to further validate and verify those ideas. 

Selecting your approach: ways to structure a win-loss analysis program

There are three types of ways that businesses tend to run their win-loss studies. Choosing which depends on the time commitment and budget.

What are the most common win-loss Models?

Everybody has their own name for them, but there are three general types and flavors of win-loss projects. 

• Retrospective (Fixed Scope)
• Continuous
• Key Accounts

The fixed retrospective approach

Also known as fixed-scope. This is when you aim to speak to 20 – for example – wins and losses that have already occurred. 

A retrospective win-loss is an excellent starting point if you haven’t run one before, that’s for a few reasons:

• Quickly hone your interviewing skills and approach
• Gain buy-in for a more rigorous, ongoing program 

This is often paired with an internal goal. 

Retrospective win-loss studies are a good way to begin your foray into voice-of-the-customer analysis. 

Within a fixed-scope win-loss analysis, you may segment your research into key segments such as:

• By buyer persona
• By Industry
• By seat-size
• By Revenue 

The continuous approach

When a deal closes, you automatically reach out to that person to invite them to take part in a win/loss interview. 

This is the gold standard of win-loss because it’s like having a security camera watching over your company’s standing in the market. If a new product starts making waves or discontent is brewing, you detect it early. 

By running a consistent program, you also unlock the ability to see trends in the data. As you make improvements to your brand positioning and sales enablement from past win-loss, you’ll begin to see the improvements in real data. 

The key account’s approach

(aka deal post mortem, strategic account study)

This is where a minority of deals are invited to discuss their purchase decision. The most common arrangement is to invite:

• Very large deals
• Early adopters of a new product
• Leads from an aspirational persona 

Tip: Fixed-scope is best for first-time projects and pilot studies

That said, you miss out on trends that you glean from continuously running pilots. Ultimately, the framework you choose depends on your budget and scope.

Running a Win-loss Program

We talked about the different ways to structure a research project (it’s the section above if you skipped here). Let’s assume you’ve chosen the fixed-scope approach. Here are the basic steps you’ll go through.

What are the 6 Steps of Running a Win-Loss Analysis?

There are common steps among all win-loss projects, regardless of the exact format you’ve chosen.

1. Review Data:

• Identify problem segments where your win-rate is significantly lower than average.
• Likewise, identify segments where the win-loss ratio is higher than average.
  • It’s interesting to interview a sampling from each to understand where their opinions and needs overlap and diverge.
• This step aims to define your outreach segments; how many wins or losses will you reach out to?

2. Get organizational buy-in:

• The findings of win-loss interviews are useless if they are left in a dusty Google Drive to gather digital dust.
• It sounds so obvious, but it bears stating anyway. To make sure results are listened to and used by relevant teams, it’s best to loop them in early and invite them to contribute to the design of the project.
• A good first step is asking each team to develop internal hypotheses or ideas about customer frustrations and business strengths.
• “If you had a room of customers, all hooked up to lie detectors, so you know they are telling the truth, what would you like to ask them?”

3. Design Interview and Survey Approach

• Based on input from the teams you consulted in Step 2, you’ll have a good basis for designing an interview that asks poignant, relevant questions that leaders inside the business actually want answers to.

Invite, Conduct, and Record Win Loss interviews and Surveys

This section could warrant an entire book. You’ll have limited time with interviewees. 30 minutes is the most common length of time that buyer interviews run.

With a tight time limit, you’ll need to carefully order and prune your interview script so that it flows naturally as a conversation but also weaves in your must-have questions early. It’s not as easy as it sounds (expect to run a few interviews to find your footing).

A toolset I’d recommend is:

• Bulk email invites using your Marketing Automation tools like Hubspot or Marketo.
• Follow-up at a reasonable amount of time.
• Use Calendly or Chili Piper to facilitate booking time with your interviewers or interview team.
• Use Zoom and/or Otter.ai to record calls. If you work in an industry with strong privacy requirements, AWS Transcribe has options to automatically remove PII though you’ll need a developer to set it up.

5. Analyze the win-loss survey and interview data

Now it’s the phase where you spend quality time with Excel. Analyzing the transcripts is a laborious task.

Some Tips:

• Break your transcripts into question-answer sections.
• Take one question, e.g., “What was your experience with the sales team like?”
• Lay out every answer from every interviewer side-by-side
  • Review each answer for keywords or key concepts.
• You’ll quickly see patterns emerge.
• Some questions won’t yield strong trends. You’ll get a range of answers. This is completely normal.
• Rinse and repeat for all questions.

6. Present back to stakeholders, gather recommendations

How Long Does a Win-Loss Program Take?

To avoid an annoying “it depends” type of answer, I’m going to answer, “how quick would the most minimalist win-loss program take.”

The first thing we’d minimize is the number of interviews you conduct.

25-30 interviews are about the minimum that I’d recommend conducting if your goal is to uncover common themes. For that volume of interviews, I’d estimate you need around four weeks.

Hypothetically, a single person could conduct eight interviews a day and book the lot on four days. With a few people, you could reduce that time to a day or two.

As we’re talking about the minimalist DIY type approach, the likelihood is that it’s just you conducting interviews. You’ll find that once you invite people to interview, people tend to prefer making a booking in the following 2-5 days. The same day is too soon. Booking weeks ahead, people tend not to show up.

The other reality is that there’s a conversion rate on interviews, which is extremely variable, too.

• How many possible interviewees do you have (lead volume)
• How recently you were working with possible interviewees
• The level of relationship you had (somebody who was working with sales for six months is more likely to interview than somebody who took a 15-minute intro call and disappeared)
• Incentives attached and reimbursement for their time
• Seniority (as a proxy for business)

Win-Loss Ratios, Statistics, and Metrics

What is the win-loss ratio calculation (in sales)?

Put your TI-83 Plus away, the calculation is very basic.

win loss ratio = number of wins ÷ number of losses

Is the win-loss ratio the same as a close rate?

Not really.

• The close rate (aka closed-won rate) is calculated as leads divided by closes (won or lost). It takes into account the leads entering the sales funnel and conclusions at the end.

• The win-loss ratio calculates how many wins versus losses occurred in a given month. It does not reference leads, which means leads that have not made a decision or take a long time to close do not affect the ratio.

When do you use the win-loss ratio?

I use the win-loss ratio to measure product-segment fit.

Businesses have a problem. Although they talk about their ideal customer or segment, they actually have a variety (why we have various personas).

Oftentimes, the most lucrative segment is dominated by larger organizations. Businesses have a few who make up a huge proportion of their revenue.

How do you measure how well your product maps to what those clients need? Support tickets? Feature requests, CSAT scores? This is where the win-loss ratio can come in handy.

Use the win-loss ratio as a simple test for product-segment fit, which is finding which segment of your current leads and customers loves your product the most. 

It’s a simple measure that ignores lead volume, time to close, the fact that larger deals have lots of seats (which create lots of support tickets), and gets down to the simple question: what proportion of deals in this segment buy, and which do not?

Other win-loss resources

• Win-loss Analysis on Wikipedia
• Our guide to running a win-loss program

Win-Loss analysis software toolbox

Email sending:

• Your marketing automation tool: Hubspot, Marketo, Pardot, etc.
• Basic tier email: Mailchimp, MailerLite

Invite and Calendar Management Options:

• Calendly: Good all-rounder with basic features
• Chili Piper: More advanced booking tool with better options for screening and routing invitees to the right form.

Call Recording:

• Zoom.us – best for most people. Multiple dial-in numbers, easy recording.

Transcription:

• Otter.ai – the best for most people
• AWS Transcribe – best if you have a dev team and/or need automatic PII removal or multi-lingual support.