Net Promotor Score (NPS): The Good and the Ugly

In 2003, Fred Reichheld, a marketing consultant at Bain Consulting, published an article in the Harvard Business Review titled:  ‘The One Number You Need to Grow,’ with a tantalizing claim that the response to this one question, converted to a simple metric called Net Promotor Score or NPS, can predict your organization’s revenue growth rate. It then made even a bolder claim that no other metric is better than NPS in doing so.

By the time research by independent observers showed that these claims were either overreaching or false, the NPS has already gained a serious foothold in many organizations including many Fortune 500 companies. Lured by the simplicity of one question and Harvard Business Review’s cache (followed by two books on the subject by Reichheld), the simplicity of a single metric that anyone can compute with a simple calculator, and the golden promise of linkage to revenue growth (never in the history of business before has there been a single metric that could predict revenue growth), NPS, despite its many flaws has grown like wildfire. As a result of high usage, it was possible to create a set of benchmarking data that other organizations could use, thus increasing the appeal of the NPS metric.

At the same time, many researchers started checking on Reichheld’s claims and found many issues with these claims as well as other aspects of NPS formulation.  This article is meant to provide a perspective on the validity and utility of NPS.

What is NPS?

Although NPS is widely known, it is useful to list it here as a reference for our discussion.

“How likely is it that you would recommend [Organization X] to a friend or colleague?”

Customers are asked to rate their answers on a 0-10 scale, which is divided up into three categories: “Detractors (0 – 6),” “Passives (7 – 8),” and “Promoters (9 – 10)”

NPS = % of Promotors – % of Detractors.

Once organizations started using NPS, they realized that they had the NPS as a number, but didn’t know what to do with it. If the number was low, they had no idea why it was low, and what actions they could take to improve it. Based on this rising criticism, Fred expanded the one question to two questions, the second question being an open-ended question asking why the respondent answered the NPS question the way she did in order to explore the reasons. Now you still had to read, code, text analyze and collate these observations to come up with some insights and action avenues.

(As a personal note, PeriscopeIQ was the first to introduce the concept of metrics taxonomy (using right metrics for right strategic objectives) and introduced its own ‘NPS’ (called Net Recommendation Score or NRS) long before there was NPS. NRS avoided many of the issues attributed to NPS.)

Issues with NPS

Some researchers question the formulation of the question itself as a speculative (How likely is that you would recommend____, rather than a behavior formulation, In the last 6 weeks, did you recommend___). Some feel that the addressed audience (__ a friend or colleague) is not comprehensive enough and maybe a family member (_ a friend, colleague or family member) should be added who is usually the first recipient of such recommendations. In our view, such criticism is minor, and the question formulation is fine, but it should be refined slightly based on the audience.

As indicated earlier, many serious researchers validly challenge the expansive claims of NPS as being the best metric to correlate to revenue growth. In fact, some researchers have shown using the same data (to the best extent possible) as used by Reinbach that a well-known metric, American Customer Satisfaction Index (ACSI) reported now for over 25 years, is as good as the NPS in making such predictions. Further, correlation is not causation. There is no doubt that high customer loyalty is one of the key factors in driving revenue growth but companies that do one thing right also do many things right to help drive the growth.

The most important issue about NPS is how NPS is calculated. The concept, although somewhat intuitive, is totally random. On a highly dispersive scale of 0 to 10, it is hard to distinguish between giving a rating of 8 to 9. But the outcome is stark: Give a rating of 9, and your individual NPS is 100. Give a rating of 8, and your individual NPS score is 0. If someone gives you a rating of 0 (Not at all likely, usually given by people who are disgusted with their experience) or a rating of 6 which is a kind of neutral rating, your individual score is still -100. These random cut-offs and large reversals, without any true scientific justification, give a fit to serious researchers.

NPS’s claims that it works for nearly every business or organization are overreaching. NPS is a metric based on voluntary recommendations that work better for some businesses than others. Businesses that are based on standardized products and direct-to-consumer sales benefit more from the use of NPS. B-to-B businesses, businesses offering complex products such as computer hardware and software require multidimensional insights (rather than only one offered by NPS) from the customers to drive success. Understanding these limitations is important.

NPS Merits

The biggest thing going for NPS is its ubiquity. NPS is now used widely, across nearly all industries and its use is spreading globally. This wide use allows access to a wide range of benchmarking data (although the data quality and poor application of such data remain problematic) that can be very beneficial for process improvements. Further, NPS has become an industry in itself, spawning consultants, trainers, conferences and seminars that allow opportunities for learning and interaction with other professionals facing similar challenges.

Despite all the challenges and concerns whether it is the best metric as claimed or not, there is a general agreement that it is a useful metric when applied properly. Considering its popularity and availability of related resources including benchmarks, it is a metric that should be considered.

NPS is simple to compute and easy to understand.

Key recommendations in using NPS

First, if your organization has been using other metrics such as customer satisfaction or likelihood to purchase in the past and you have found those to be useful, there is no need to get rid of those metrics and replace those with NPS. As indicated earlier, researchers have found that some of these metrics are as good or better for predicting desired outcomes. You may want to add NPS as an additional question to the survey if your management insists its addition for the sake of benchmarking. If you are conducting your study properly, you will see a high correlation between NPS and other metrics anyhow.

One of the biggest and highly valid criticism of NPS is that it is a lone question, it tells ‘what’ but not ‘why’. The addition of the open-ended question is useful, but the actual experience shows that many respondents don’t leave any comment and those who do require detailed review and categorization to make much sense of the data. Text analytics software is useful but not highly reliable despite all the advances in text analytics and is likely to remain so considering the wide variance in style and content of the comments. We recommend a 4 to 6 question survey (less than 2 minutes of expected completion time) which includes a few structured questions focused on the underlying reasons for the response. It would also be a desirable to include a satisfaction question, and an open-ended question for general comments. We like the recent trend in NPS surveys where the NPS question is included in the invitation email message, and a response to the question leads to an expanded survey which respondents may choose to respond to it or not.

NPS benchmarks are useful for knowing where your organization ranks relative to others, but an obsession with benchmarking can be more harmful than useful. First, one does not know how valid and reliable the reported benchmark data is as it is rarely accompanied by reliable statistics to assess its validity. Second, such benchmarks vary substantially by the type of industry, and whether the organization’s product is a physical product or a service. For example, NPS of a consulting company or a membership association should not be benchmarked against NPS for retailers. So, if you are benchmarking yourself, make sure that you do so with reliable benchmarks for comparable industries.

Much more important than external benchmarking is internal benchmarking by setting your own goals every year and then trying to meet them. Amazon really set the stage for obsession with customer excellence but never looked for and still does not look for external benchmarks. External benchmarking can be helpful if your scores are very low and then deciding what your internal target should be for the next year. With external benchmarking, there can also be smug satisfaction if your scores are comparable to the averages, but it could hide some serious pitfalls.

– One of the key scientific arguments against NPS, as indicated earlier, is illogical scoring (where the value of response of 6 gets the same score as a response of 0) to compute NPS. Some organizations separate out the % count of those who select a response value from 0 to 2 and call them extreme detractors or some other similar nomenclature. These organizations report both NPS and % extreme detractors (who are highly dissatisfied and are much more likely to bad mouth you). If only one metric is to be used, we recommend a separate benchmark (call it Modified NPS) where extreme detractors are scored as -2 (instead of -1). So, the Modified NPS would be = NPS – % extreme detractors. Obviously, only NPS should be used for benchmarking but the focus on extreme detractors is important which under the normal NPS schema gets washed out along with other detractors.

– NPS is a loyalty metric and it is not the same as a transactional satisfaction metric. Loyalty is built based on experience with multiple transactions and experiences, and similarly, it typically takes more than one bad experience to create a serious dent in loyalty. For example, one bad experience with returning an item to Amazon is not going to change how likely are you to recommend Amazon to your friends. However, if asked about the satisfaction with a particular transaction, the response may be highly negative and a drastic shift from the experience with a previous transaction. That’s why it is important to understand the relationship between loyalty and satisfaction. Normally there is a direct correlation between loyalty and satisfaction but there are instances when it is not. Understanding and properly dealing with those instances is very important to build a culture of customer excellence which will yield both higher satisfaction and NPS metrics. Unfortunately, we have seen that the NPS is now widely used for measuring transactional satisfaction which can lead to many invalid responses.

– And, finally, NPS, despite its popularity, is not really the best metric for all industries. This is specifically true for B2B organizations with infrequent transactions of moderate or high value. The buyers here can’t reliably answer the NPS question as they are not going out and recommending an organization to their colleagues and friends. It is much better to ask a question about their satisfaction with a particular transaction and all its attributes (quality, timeliness, price, etc.) than ask the NPS question (this is also the reason that a properly designed satisfaction metric measured over a long period has a much wider applicability and is a more reliable metric for desirable outcomes). We have seen that the NPS question is blindly used for nearly every occasion. If the NPS question must be used for a certain reason, we suggest modifying question to: If asked, how likely are you to recommend …