Guides / customer measurement
Customer satisfaction and NPS
Net Promoter Score is quoted to one decimal place, tracked monthly and used to set bonuses. It is also one of the noisiest measures in common use, roughly half again as variable as the plain percentage it is built from. This guide works an NPS out with its confidence interval, shows how large a change you can actually detect, and sets out when CSAT or CES tells you more.
Ask how likely someone is to recommend you on a 0 to 10 scale. Scores of 9 and 10 are promoters, 7 and 8 are passives, and 0 to 6 are detractors. NPS is the percentage of promoters minus the percentage of detractors, reported as a whole number between minus 100 and plus 100. The calculation is trivial. The part almost everyone skips is the confidence interval, and without it a tracked NPS is mostly measuring its own sampling error.
Definitions
The three measures, and what each asks
Three metrics dominate customer measurement. They are not interchangeable, and the most common mistake is using one to answer a question another was designed for.
| Measure | The question | Scale | Answers |
|---|---|---|---|
| NPS | How likely are you to recommend us? | 0–10, collapsed to three groups | Overall relationship, loosely |
| CSAT | How satisfied were you with this? | 1–5 or 1–7, reported as % satisfied | A specific interaction, just now |
| CES | How much effort did this take you? | 1–7 | Friction in a process |
NPS asks about intention, not behaviour, and intention to recommend is a weaker predictor of actually recommending than its popularity suggests. CSAT is tied to a moment and decays fast, so asking two weeks later measures memory. CES is the narrowest and often the most actionable, because reducing effort is something a team can be asked to do.
NPS is the only one of the three that throws away the scale it collected on. An 11-point response is compressed into three categories, two of which are then subtracted from one another. That compression is where both its simplicity and its statistical trouble come from.
Worked example one
NPS with its interval
400 customers respond. 180 score 9 or 10, 140 score 7 or 8, and 80 score 0 to 6.
Step 1. The three proportions
promoters = 180 / 400 = 45%
passives = 140 / 400 = 35%
detractors = 80 / 400 = 20%
Step 2. The score
NPS = 45 − 20 = +25
Reported as a number, not a percentage, which is a convention worth keeping because it reminds everyone it is a difference rather than a share.
Step 3. The standard error
NPS is a difference between two proportions drawn from the same sample, so its variance has a specific form:
Var = [ pₚ + pₔ − (pₚ − pₔ)² ] / n
= [ 0.45 + 0.20 − 0.25² ] / 400
= 0.5875 / 400 = 0.001469
SE = √0.001469 = 0.0383, so 3.83 points
Step 4. The interval
95% CI = 25 ± 1.96 × 3.83 = +17.5 to +32.5
A span of 15 points. The organisation reporting "our NPS is 25" has a number that could reasonably be 18 or 32.
| Sample size | NPS | 95% CI | Width |
|---|---|---|---|
| 100 | +25 | +10.0 to +40.0 | 30 points |
| 400 | +25 | +17.5 to +32.5 | 15 points |
| 4,000 | +25 | +22.6 to +27.4 | 5 points |
Identical score, three very different levels of knowledge. A monthly NPS from 100 responses is not a measurement, it is a wide guess that happens to be expressed as a whole number.
The structural problem
Why NPS is noisier than a percentage
Compare like with like on the same 400 responses.
SE of the promoter percentage alone = √(0.45 × 0.55 / 400) = 2.49 points
SE of NPS = 3.83 points
NPS is 1.54 times noisier than simply reporting the promoter share, on exactly the same data. There are two reasons, and neither is a flaw in how you collected it.
This is why a tracked NPS bounces. Teams interpret the bouncing as customer sentiment responding to their work, when a large part of it is arithmetic.
Worked example two
How big a change can you actually detect?
The question that should be asked before any NPS target is set, and almost never is.
Detecting a change needs the uncertainty of two waves, not one, so the standard error of the difference is larger again by a factor of about 1.41.
| Responses per wave | Smallest NPS change you can reliably detect |
|---|---|
| 400 | 15.2 points |
| 1,000 | 9.6 points |
| 2,500 | 6.1 points |
| 5,000 | 4.3 points |
| 10,000 | 3.0 points |
Read the first row carefully. With 400 responses a wave, which is a perfectly respectable survey, nothing under about 15 points is distinguishable from noise. An organisation celebrating a rise from 25 to 31 has observed nothing.
To track NPS monthly at a useful resolution you need thousands of responses every month. Most organisations running monthly NPS have a few hundred, which means the chart on the wall is substantially a picture of sampling variation.
If a bonus depends on moving NPS by five points and the survey can only detect fifteen, the bonus is a lottery. Work out the detectable change before the target is agreed, and set the target above it. This is a conversation to have with the board once, early, rather than every quarter afterwards.
Worked example three
Breaking NPS down by segment
The first thing anyone does with an NPS is split it. By customer size, by tenure, by region, by product. This is where the noise problem stops being theoretical, because every split divides the sample.
The same 400 responses, cut five ways.
| Segment | n | NPS | 95% CI | Width |
|---|---|---|---|---|
| Enterprise | 50 | +44 | +24.7 to +63.3 | 39 points |
| Mid-market | 110 | +29 | +15.0 to +43.2 | 28 points |
| SMB | 160 | +25 | +13.3 to +36.7 | 23 points |
| New, under 6 months | 47 | +9 | −15.0 to +32.0 | 47 points |
| Legacy | 33 | +6 | −20.4 to +32.5 | 53 points |
| Overall | 400 | +25 | +17.5 to +32.5 | 15 points |
The story the table appears to tell
Enterprise customers love us at +44. Legacy customers are unhappy at +6. That is a 38 point gap and someone is about to build a retention programme around it.
The story the intervals tell
Enterprise runs from +24.7 to +63.3. Legacy runs from −20.4 to +32.5. Those ranges overlap. On this data you cannot say with confidence that Enterprise customers are more positive than Legacy ones at all.
The apparent 38 point gap is built on 50 responses against 33. Neither number is a measurement; both are wide guesses that happen to differ.
Why the smallest segments always look most extreme
Look at the widths. Legacy, with 33 responses, has an interval 53 points wide, more than three times the overall figure. Small samples produce extreme values by chance, so when you rank segments the smallest ones reliably appear at the top and the bottom. That is a property of the arithmetic, not of your customers.
It is the same mechanism that puts tiny departments at both ends of an engagement league table, and the same reason mixed effects models shrink small-group estimates toward the average.
Decide the segmentation and the minimum cell size before the data arrives, and write both into the analysis plan. Choosing how to cut after seeing the numbers is how a business ends up investing in a segment difference that was never there.
Information loss
What NPS throws away
Three specific distortions follow from compressing eleven points into three groups.
| Customer movement | Real change | Effect on NPS |
|---|---|---|
| 6 → 8 | Meaningful improvement | None. Detractor becomes passive; the score rises only through the detractor share. |
| 7 → 8 | Small improvement | None at all. Both are passives. |
| 8 → 9 | Small improvement | Same impact as a customer moving 0 → 9. |
| 0 → 6 | Enormous improvement | None. Still a detractor. |
A programme that moves a lot of customers from 5 to 8 has done real work and will show almost nothing. A programme that nudges a handful from 8 to 9 will show a clear rise. The metric rewards movement across two arbitrary boundaries and ignores everything else.
The mean of the 0 to 10 scores uses all the information and is far more sensitive to genuine change. It is less memorable, which is the only serious argument against it.
Choosing
When CSAT or CES tells you more
| You want to know | Use | Why |
|---|---|---|
| Whether a specific interaction went well | CSAT | Tied to the event, asked immediately, easy to act on |
| Where a process is painful | CES | Points at friction, and effort is something a team can reduce |
| Overall relationship, tracked slowly | NPS, annually | Its noise is tolerable over long intervals with large samples |
| Whether sentiment actually moved | Mean of the 0–10 | Uses the whole scale, far more sensitive than the collapsed score |
A reasonable arrangement for most organisations: CSAT or CES at the point of interaction for operational feedback, and NPS once or twice a year with a large enough sample to mean something. Monthly NPS on small samples combines the worst of both.
Whichever you use, the same self-selection problem applies. Customers who respond to a survey are not a random sample of customers, and the ones who feel strongly answer first.
Doing it
In R, SPSS and Stata
R
No package needed. The variance formula is three lines and the transparency is worth more than a dependency.
nps_ci <- function(prom, pass, det) {
n <- prom + pass + det
pp <- prom/n; pd <- det/n
nps <- (pp - pd) * 100
se <- sqrt((pp + pd - (pp - pd)^2)/n) * 100
c(nps = nps, lower = nps - 1.96*se, upper = nps + 1.96*se)
}
nps_ci(180, 140, 80)
For the mean of the raw 0 to 10 instead, t.test(score) gives the interval directly, and it will be narrower than the NPS one on the same data.
SPSS
Create the grouping, then tabulate.
RECODE score (9 THRU 10 = 1) (7 THRU 8 = 2) (0 THRU 6 = 3) INTO npsgroup.
VARIABLE LABELS npsgroup 'NPS category'.
VALUE LABELS npsgroup 1 'Promoter' 2 'Passive' 3 'Detractor'.
FREQUENCIES VARIABLES = npsgroup.
SPSS will not produce an NPS interval. Take the promoter and detractor percentages from the frequency table and apply the formula above, or use the bootstrap module on a computed NPS variable.
Stata
recode score (9/10 = 1 "Promoter") (7/8 = 2 "Passive") (0/6 = 3 "Detractor"), gen(npsgrp)
tab npsgrp
gen nps_i = 100 * (score >= 9) - 100 * (score <= 6)
ci means nps_i
That last trick is the neat one. Scoring each respondent as +100, 0 or −100 makes NPS an ordinary mean, so ci means returns the correct interval directly and every standard command works on it.
Some platforms collect 1 to 10 rather than 0 to 10, and some reverse the scale so 0 is best. Applying the standard 9–10 and 0–6 cut-points to a reversed or shifted scale produces a plausible-looking number that is completely wrong. Tabulate the raw responses before recoding.
Avoid these
Six mistakes that cost you credibility
| Mistake | Why it matters |
|---|---|
| Reporting NPS without an interval | At 400 responses an NPS of 25 spans 18 to 33. The bare number implies a precision it does not have. |
| Tracking it monthly on small samples | With 400 a wave, nothing under 15 points is detectable. The monthly chart is largely sampling noise. |
| Setting targets below the detectable change | A five point target on a survey that resolves to fifteen makes the bonus a lottery. |
| Comparing against published benchmarks | Different wording, different industries, different response rates, and no interval on their figure either. |
| Treating it as a measure of behaviour | It records stated intention to recommend, which is a weaker predictor of recommending than its reputation suggests. |
| Ignoring who did not respond | Unhappy customers often leave rather than answer. A rising NPS can mean detractors churned. |
NPS has a wider interval than almost anyone reports, so a three-point move on a few hundred responses is usually noise presented as a trend. Send the responses and you get the score with its interval, and a straight answer on whether it moved.
See survey and insight analysis
Saad TariqPrincipal StatisticianWriting it up
How to report it
The sentence worth drafting carefully: "NPS was +25 (95% CI +18 to +33) from 400 responses, a 31% response rate. At this sample size changes smaller than about 15 points cannot be distinguished from sampling variation."
Common questions
Frequently asked questions
How do you calculate NPS?
Subtract the percentage of detractors, who score 0 to 6, from the percentage of promoters, who score 9 or 10. Passives scoring 7 or 8 are excluded. The result runs from minus 100 to plus 100 and is reported as a whole number rather than a percentage.
What is the confidence interval for NPS?
Its variance is [p promoters + p detractors minus the squared difference between them] divided by n. With 180 promoters, 140 passives and 80 detractors from 400 responses, NPS is +25 with a standard error of 3.83 and a 95% interval of +17.5 to +32.5.
Why is NPS noisier than a normal percentage?
Because it subtracts two proportions that each vary, and because it discards the passives entirely. On the same 400 responses the promoter percentage has a standard error of 2.49 points while NPS has 3.83, so NPS is about 1.5 times noisier on identical data.
How many responses do I need to track NPS?
More than most organisations collect. At 400 responses per wave the smallest reliably detectable change is around 15 points. Detecting a 5 point change needs roughly 5,000 per wave, and a 3 point change needs about 10,000. Monthly NPS on a few hundred responses is largely measuring its own sampling error.
What is a good NPS?
There is no universal answer, and published benchmarks are weaker than they look because question wording, industry, channel and response rate all differ and none of them comes with an interval. Your own trend, measured consistently with a large enough sample, is more informative than any comparison.
Should I use NPS, CSAT or CES?
CSAT for a specific interaction, asked immediately. CES where you want to find friction in a process, because effort is something a team can be asked to reduce. NPS for an overall relationship tracked slowly with a large sample. For detecting genuine movement, the mean of the raw 0 to 10 beats all three.
Can a rising NPS be bad news?
Yes. If unhappy customers churn rather than respond, they leave the sample and the score rises while the business worsens. Always read NPS alongside retention and the response rate, not on its own.
Keep reading
Related
Employee engagement survey analysis
The same problems inside the organisation: response rates, intervals and league tables.
Self-selection bias
Why the customers who answer are not the customers you have.
Survey and insight analysis
Survey data analysed and reported so the findings hold up to scrutiny.
Related services
If the findings have to persuade a board
Survey and insight analysis
Survey data analysed and reported so the findings hold up to scrutiny.
Fixed quoteStatistical consultancy
Design, analysis and reporting for charities, universities, the NHS and business.
From £950Data cleaning & preparation
Messy data made analysis-ready, with a documented log of every change.
Fixed quoteEvery piece of work is done and signed by a named statistician. Send the data or describe the project and you get a price and a deadline, usually within one working day.