“Anecdotal Data” is an Oxymoron

By: Beth Barrett, Senior Content Director

Everyone likes an anecdote. They can lighten the mood, bring life to the mundane, and occasionally make recurring meetings more entertaining. However, we get nervous when we hear the words “based on anecdotal data…” By definition, the term is an oxymoron, and the information that follows is rarely representative or substantiated.

Nevertheless, far too many organizations approach anecdotal information as something close to gospel truth, even (or especially) when its source is elsewhere in the company. Real-life examples of this phenomenon abound:

This is akin to basing parenting decisions on conversations your child reported having with other children: interesting, perhaps, but more indicative of incongruent incentives or lack of information than anything else. With respect to a company, internal claims are often not representative of the larger customer base, and can lead organizations rushing down a path to change without a complete understanding of “the truth.”

THE TRUTH LIES IN THE DATA

Understanding your data means knowing what customers really think of your brand, product(s), and service(s), and making informed, data-driven decisions to improve those outcomes.  In practice, Hanover works with clients across industries and specialties to uncover the truth behind the one-off stories – the anecdotes.

Here is an anecdote to illustrate the point (irony notwithstanding).

One technology provider in the healthcare space heard from its customer support team that a few people recently canceled their service because of usability issues with part of the interface. Upon hearing this feedback, the Product Manager was ready to leap into action; “We must immediately re-design the user interface!” It never occurred to the PM that the support team’s feedback might be biased or incomplete.  However, we questioned the hypothesis immediately. 

Hanover’s approach to this task did not require collecting a single piece of feedback from customers. Instead, using a more scientific method, we looked at the client’s internal data to conduct a key driver analysis that examined the past two years of service cancellation data.  This enabled us to create a model that predicts service cancellation based on a sequence of indicators and to uncover the reality of our client’s situation.

It turned out that the primary driver behind cancellation is the timeliness of customer service’s response to a few key service issues – not platform usability.  It may not be a surprise that the customer support team answered as they did: self-preservation is a powerful motivator. In contrast, data analysis helped the company reach the right answer, avoiding wasted time and resources and facilitating future efforts to the true root cause of the problem.

Anecdotes help bring ideas to life, but important decisions should be based on data.