Mark Twain popularized the saying that, “There are three kinds of lies: lies, damned lies, and statistics.”
Numbers seem objective. A chart seems authoritative. But most people’s numeracy and statistical knowledge is quite low. Which unscrupulous communicators can take advantage of intentionally while the uninformed do so accidentally.
Seeing Trends Where They Don’t Exist
Take the chart below as an example. Imagine this is the customer satisfaction (C-Sat) data for a startup where you’ve just started working and the CEO and founder pops this graphic up in an all-company meeting to talk about the company’s fantastic momentum. He finishes by saying, “…we can see that 2021 was a great year with customer satisfaction on a definite upswing.”
There are at least two problems with this chart:
- By displaying annual data as a line graph the chart implies continuity between data points. That is, the chart implies that we could learn what the company’s C-Sat was halfway through 2019 by simply finding the right point on the graph. This is incorrect. The data represents annual measurements so should be displayed as a bar chart or scatter (just dots) graph.
- More importantly, the chart does not show any reason to believe “customer satisfaction [is] on an upswing.” 2021 C-Sat increased vs. 2020, true. But you cannot derive any sense of momentum or trend from this data. Notice that in 2017 C-Sat was just about the same as 2021. From the chart we see that there is variation in the company’s customer satisfaction and that no one should be surprised if the number falls anywhere between 65% and 85%.
The CEO might be looking at a slight upward ‘trend’ starting at the low point of 2018. But 4 data points do not make a trend, at least with the information we have in this chart. The CEO should probably be challenging the company to figure out why these numbers are consistent – steady at 75% for instance – rather than claiming random variation as a success.
Warning!
This sort of selective storytelling with data is often perpetrated by entrepreneurs and senior leaders. Sometimes the reasons are innocent – the eternal optimism necessary to succeed. Often the reason is more insidious: a conscious effort to manipulate data to arrive at a predefined outcome. Regardless, knowledge workers need to apply critical thinking skills and resist this corruption of logic – diplomatically. But would you really want to work for a company that manipulates facts to tell the stories they want?
Using Y-Axis Values to Lie
One of the most egregious and amateurish manipulations involves changing the values of the Y-axis. It’s so easy to identify that you have to wonder why anyone tries.
Back to our poor fictitious CEO. Now he’s trying to tell his dedicated troops that revenue is WAY up and uses the following chart. The Y-axis starts at $1.250,000 – making the growth in revenue look bigger than it actually is.
Here’s the real data – still significant and constant growth but not as dramatic as the green chart. In the blue version the Y-axis starts at zero.
The same strategy can be used to minimize bad news. Our CEO moves on to discuss marketing spend – the cost to acquire customers. He shows the following chart – designed with an inflated maximum Y-axis value to show a gentle growth in marketing costs.
With a more sensible Y-axis maximum, so that our data fills about ⅔ to ¾ of the vertical space, we get a completely different message. Although it isn’t entirely negative – marketing costs seem to be leveling off.
In fact, our CEO is actually leaving good news out of his presentation: compared to revenue, marketing was growing more quickly in the first 3 years represented on the chart, but has levelled off, while revenue continues to grow.
There are other versions of the same game – stretching out the width of a chart to minimize change or increase its height to exaggerate difference. The two charts below show the exact same data but communicate far different messages.
There are many variations to the simple games we’ve outlined above. Another favorite amongst unethical communicators is to bury data in complexity – include so many lines and axes and legends that your audience listens to your interpretation rather than think through what they’re seeing for themselves.
In our final example – a chart that should get anyone other than an intern fired – we use a left and right axis to hide the marketing costs by making them appear small compared to revenue.
So what should you do?
- Never accept the default values provided by Excel or Google Sheets.
- Use your judgement to adjust Y-axis maximum to show your data filling approximately 75% of the vertical space.
- Think twice before making the minimum value of your Y-axis anything other than zero. You need a very good reason to change this.
- Think about the real message of the data you’re showing. Hopefully you have a deeper understanding of the subject matter than the information in just one chart and can tell the story ethically.
- Deny the temptation to tweak the overall size of your chart to exaggerate the story you want your audience to see.
- When leaders in your organization try to tell stories that aren’t supported by the data try to coach them – in private – to bemore data literate. Like you!
