Customer experience quality in the U.S. has just hit an all-time low. And, in related news, customer experience quality in the U.S. has also hit an all-time high.
Let the confusion begin.
These two headlines are real. The first is from Forrester Research’s 2024 “U.S. Customer Experience (CX) Index” study, released on June 17, 2024. The second is from the Q1 2024 “American Customer Satisfaction Index (ACSI),” released on May 7, 2024.
These are two of the most widely cited national studies that gauge how U.S. consumers are feeling towards the businesses they patronize. Released just about a month apart, they draw two very different conclusions about the current state of consumer satisfaction. But wait, there’s even more intrigue here…
The chart below shows the 9-year trend for each measure.
Notice anything interesting about these two line graphs? That’s right – they are mirror images of one another. From 2016 through 2024, these two vaunted measures of customer experience moved in lockstep OPPOSITE directions.
Granted, the two studies use different methodologies for scoring customer experience: According to the companies’ descriptions of their respective studies, the Forrester Index focuses on “effectiveness, ease, and emotion” whereas the ACSI focuses on “customer expectations, perceived quality, and perceived value.” But to the casual observer, those distinctions are meaningless.
What business leaders see is an industry that can’t get its story straight: Is customer experience quality at an all-time high or an all-time low? Apparently, it wholly depends on which Customer Experience Index you’re citing – which isn’t a good look for the CX industry.
Below are three important lessons to be learned from the curious inverse correlation between the Forrester and ACSI indices – lessons that should influence not just how people interpret these closely-watched measures, but also how they gauge customer experience quality within their own businesses:
1. Customer satisfaction is a poor measure of customer engagement.
One subtle but potentially important difference between these two indices is that Forrester’s measure explicitly targets the customer’s emotional reaction to the business encounter, whereas the ACSI appears to be more focused on customers’ rational evaluation of the interaction (relative to their expectations).
That distinction matters – because, ultimately, what really drives customers’ willingness to purchase (and repurchase) from a business isn’t their logical assessment of the encounter (delivery vs. expectations). More influential is the emotional resonance of the interaction – how it made the customer feel.
The ACSI might have a blind spot in this regard, though it’s admittedly hard to imagine that alone accounting for the near-perfect mirror image trend between the two measures.
2. How and what you measure makes all the difference.
Both Forrester and ACSI use proprietary formulas to calculate their indices. Read the methodology overviews for either of them and you’ll see cryptic references to “weighted algorithms,” “multi-equation econometric models,” and “multivariable measurement components.” There isn’t a whole lot of transparency around these calculations, so it’s hard to know if the formulas themselves could account for such disparate results.
The industries covered by these firms’ consumer surveys can also make a difference. If one index has a much greater representation from durable goods companies, while the other is more focused on the service sector – that could drive variations in the results. Fact is, however, the industries targeted by the two indices are actually quite similar.
Even the survey wording and response scales used for these indices (or any Voice-of-the-Customer metric) can sway the results. A misplaced conjunction, an imbalanced survey scale, and even the order in which questions are presented – these can all materially influence the customer feedback scores.
Unfortunately, given the secrecy that shrouds the Forrester and ACSI measures, we’ll probably never know what exactly is driving the divergence in their results. Nevertheless, the key lesson to take away is that even subtle differences in the design and deployment of a customer survey can have a surprisingly significant impact on the resulting insights.
3. Changes in scores may not reflect real changes in customer experience quality.
Business performance measures should always be viewed through the (often overlooked) lens of statistical significance. Metrics, including the Forrester and ACSI indices, will rise and fall over time – but if those changes are not statistically significant (i.e., they exceed the normal, mathematically expected variation in the measure), then they’re meaningless.
Businesses routinely rejoice when they observe an improvement in their customer experience metrics. But if a month-over-month increase isn’t statistically significant, then there’s really nothing to celebrate. Conversely, executives browbeat their teams when they witness a decline in these measures – but, again, if the deterioration is within the window of expected variation, then nothing has really changed in the underlying environment, and no one deserves to be scolded.
It’s worth noting that over the last nine years, the Forrester and ACSI measures have tracked within a rather narrow band. Creatively scaled Y-axes can make the year-over-year trends look seismic, but in reality, both measures during that period fell within a tight band of their 100-point scales (2.7 points for the Forrester Index and 4.5 points for the ACSI). While both companies assert that even those small changes are statistically significant ones, the important lesson is simply this: It’s a dangerous game to draw conclusions from performance metric trends without evaluating those changes through a statistical lens.
So, these key learnings aside… who’s right and who’s wrong? Is Forrester’s view of the world more accurate than ACSI’s? Is customer experience quality at an all-time high or an all-time low?
If the mirror image graphic presented earlier tells us anything, it’s that these nationwide, cross-industry customer experience indices can, at best, be ambiguous and, at worst, be completely contradictory.
So, if you want an answer to the question “What is the state of customer experience in America today” – consider these CX indices in the context of your own customer experiences (and those you hear about from friends and family). Reflecting on the companies you regularly patronize, from Fortune 100 firms to small, local businesses, ask yourself:
- Do they consistently do what they say they’re going to do – calling you back when promised, providing information you’ve requested, etc.?
- Do they respect your time and avoid wasting it – or do you find yourself perpetually waiting in call queues and checkout lines?
- Is it easy to find someone when you need help (online, over the phone, or in a store)? Someone who’s competent, courteous, and inspires your confidence in the business?
- Do the businesses’ employees take ownership for assisting you, or do you find you’re passed off from one department to another?
- Do they communicate clearly with you, conveying information in a simple, easy-to-understand manner that resolves all of your questions, rather than triggering new ones.
- Do they anticipate your needs and treat you like an individual – or do they seem to view you as just another revenue source?
- Do they advocate for your best interests? Or do they only seem to be interested in lining their own pockets?
- Do your interactions with them elicit positive emotions such as happiness, serenity, and confidence? Or do you instead experience frustration, confusion, and doubt?
- Overall, do you feel better after you’ve interacted with these businesses as compared to before? Do they enrich you, or do they exploit you?
Ponder these questions and what you’ll probably come to realize is that you don’t really need a professionally-produced nationwide CX index to clue you in to the current state of customer experience… because you knew the answer all along.
[A version of this article originally appeared on Forbes.com.]
Jon Picoult is founder of Watermark Consulting, a customer experience advisory firm that helps companies impress customers and inspire employees, creating raving fans that drive business growth. Author of “FROM IMPRESSED TO OBSESSED: 12 Principles for Turning Customers and Employees into Lifelong Fans,” Picoult is an acclaimed keynote speaker, as well as an advisor to some of world’s foremost brands. Follow Jon on LinkedIn or Instagram, or subscribe to his monthly eNewsletter.