09 Oct Net Promoter Score: the sensitive new age guy of measurements
Maybe you’ve thought about (or even started) measuring your Net Promoter Score on a regular occasion as part of your business KPIs. But are you getting an accurate measure?
If you aren’t familiar with the Net Promoter Score measure, it works like this: in a survey tool we will ask the question ‘How likely are you to recommend [insert brand/product/service here] to family, friends or colleagues?’ The scale is from 0 – Not at all likely to 10 – Extremely likely. The Net Promoter Score is then calculated as % Promoters (those rating 9 or 10 on the scale) minus % Detractors (those rating 0 to 6 on the scale). Those rating 7 or 8 are classed as ‘Passively Satisfied’. (Read more about the basics and history of NPS here.)
Net Promoter Score (NPS) is often debated. While there are those who swear by this measure, there are others who feel it is unreliable as a measure of performance. The most common objection raised is the way it segments the sample.
Why do those who gave a rating of 6 get classed as a detractor? It is a positive score right? Well, think about the last time you got service that you gave a neutral rating or only a 6 out of 10. You might be thinking something like ‘It was OK, but it was nothing to write home about.’ As soon as you are neutral or saying things like ‘It wasn’t the best’ you are detracting from the brand, or as soon as you‘re thinking ‘I have this list of things that could have been improved’ even if the rest of your experience was pretty good, those improvements are detracting from the brand for you. When you find a provider that does it better, you will probably shift to them, especially if the reason for the lower score is due to inconsistent performance!
Why do those who rate 7 or 8 get rated as ‘passively satisfied’? Again, think about the last time that you gave this score. Did you then go and rave to all your friends about how wonderful your experience was? Probably not.
The upshot is, unless you are mad about your experience you are just not likely to talk about it or you are likely to say things like ‘it was OK’ – hence, you are not a promoter in the true active sense of the word. But of course, my experiences alone don’t make for validation of a tool and as we will see, NPS should be treated within context (there is some good research out there to show that it is relatively reliable but should only be used with a range of other metrics).
Some people tailor this question or use the NPS calculation on things that are not related to actual promotion, such as satisfaction (we don’t recommend doing the latter). The important things to keep in mind are that:
- The score is about promotion – likelihood to recommend or promote the brand/product/service.
- The score is the net promotion – not the gross promotion which is the percentage of all people recommending, but the score you get once you take away the effect of those who are detracting from your brand/product/service.
Applying the term NPS to anything else is just misleading.
OK, so now we know a little bit about this measure and we have decided that it’s an appropriate measure, how do you work out if you should administer it for your business? Before you start using NPS, there are a few things to consider that will help you to make the most of using this KPI.
Is NPS even relevant?
I can’t tell you how many times I’ve filled out a survey only to get asked this question and it’s just not relevant! Why? Because if you are the ONLY option of provider in this field, why would I recommend you to others? (I don’t want to mention any brands in particular but if you’ve ever sent a letter or received a service from the government, you can see where I’m coming from.)
Is your brand or product or service even something that people ACTUALLY recommend to others? If you have a look at some of the internet posts that poke fun at this topic, you’ll quickly get an idea of the things that people are not recommending to others because it’s just not something that they talk about. Am I going to recommend your brand of laxatives to my colleagues? Oh god no!
OK, so what should you do if you fall into this category where NPS is not quite right for you? Well, this is the only time that we recommend adjusting the questioning to something along the lines of ‘How likely are you to speak positively to others about [insert brand/product/service]?’
While this change of wording means you won’t be able to compare your NPS to that of other companies (not that this is always recommended – see the point below), at least you can get a score and it is relevant. Experience does tell us that switching to or using this measure is likely to reduce the actual score you receive, however it is a far more accurate reflection of your performance – just as long as you follow the advice under this next point.
Keep those Net Promoter Scores in context
The term ‘score’ in the name should give you a clue that this is an indicator. It is most reliably used to show trends over time and indicate whether more people are willing and likely to recommend you than people who are telling others negative things about your brand/product/service.
Alternatively, it may be telling you more about brand confidence than actual detraction. If they have just seen a terrible news story about your brand or product, they will be just as unwilling to recommend what they may feel is an unpopular opinion. Or a whole host of other things may be affecting people’s likelihood to actually recommend you beyond just personal experience, as discussed above.
The score is not telling you anything more or less than how you are tracking in terms of recommendations to others. It needs to be taken in full context of what you offer, where you are in the market, who your consumers are, what confidence there is in your product and so much more – so don’t rely solely on NPS! It is just your car’s indicator, not your whole car.
When I say 66% of your customers are promoters, I am giving you a tangible fact about the world (hopefully with some reliable margin of error). When I say your NPS is +45.3, it only means that more people will promote you than say negative things about your offering. It only has some context if you have some other information, like last time your NPS was +36.7 (improvement, keep on going with that new initiative!) or last time it was +78.5 (Hmmm, looks like something has gone wrong…). Keep in mind also that the NPS is also way more sensitive than a normal measure so changes tend to be large – hence why I labelled it the ‘sensitive new age guy’ of all measures. This is because of the way it is calculated meaning that any change in promotion OR detraction will affect the score. So should you be worried/excited that your NPS changed by a couple of points? Probably this is just normal variation (for those hardcore stats nerds, you can read about how to calculate margin of error for your NPS here, which will obviously allow you to run significance testing).
A benchmark measure is important to give that context, which is where a market research specialist can help, designing a reliable tool that includes a host of measures and then allowing you to track it over time.
Speaking of benchmark measures, I often get told things like ‘I read on a website that the industry average is +60, but our NPS is only +20! What are we doing wrong?’
Well, there are a few things to keep in mind. Firstly, do these competitors do EXACTLY what you do? It’s not worthwhile to compare NPS if their offering varies from yours in any marked way and never compare across industries – as discussed above, there are a whole stack of reasons that are industry related that will make NPS differ because people are just more likely to recommend certain things to people (accommodation, hospitality, etc.) and less likely to recommend other things (personal products, utility providers, government services, etc.). Secondly, people with lower NPS generally don’t post it on public websites so these industry estimates are generally very inflated. Thirdly, how did they actually measure NPS? The next point shows why this is such an important consideration.
Think about when and how this data is collected
Case study in point: Recently we took on a new client as part of brand health monitoring. They had been undertaking some measures as part of their own program of research with customers, including measuring NPS.
When our results came in through random sampling however, they were shocked to see the NPS appeared to have dramatically dropped!
The reason was the way in which we sampled. They had been using only in-store customers to measure NPS. This might seem like a good idea but there was no independence of results and the most frequent customers were the ones contributing to the score. What this meant was that those who were MOST favourable and likely to promote were the ones actually giving feedback.
When we measured, we used a random sample of everyone who had shopped there recently. The independent sampling meant we engaged with all types of customers and they all had only one say because the sampling was performed at a singular point in time.
So it’s important to make sure that your use of NPS is correct. Think also about how you present the scale as this has to always be in the same format across surveys – other presentations could change the possible result due to respondent biases and may mean that you affect the reliability of your own results.
We’ve also seen some companies actually prime their participants to how the scale works! This will artificially inflate the scores seen for NPS, which is great for you to reach that KPI but not so good if you genuinely want to understand your consumers. See the example below of a scale that is really indicating to participants what they should select:
(Another reason also not to compare your NPS too closely to that of other companies – you don’t know who is cheating!)
The other reason that you may end up with skewed results is the placement of the question in your survey tool. Survey design is, however, a totally different essay for another time.
So how likely are you to recommend this article to friends, family or colleagues? (I’m kidding, don’t answer that.)
Want to find out more about how newfocus can help you measure Net Promoter Score? Contact us here