The Vicious Cycle of Data Impoverishment

It is not difficult to present data in clear, accurate, and meaningful ways. The skills that are required to do this can be easily learned. They involve simple principles, most of which have been taught for many years. So why are most data displays so impoverished?

One reason is that we live in a time when people use their computers as a replacement for skills. If you rely on Excel or almost any other software tool that creates charts to do the work for you, then your data displays will fail miserably. Effective data visualization practices are only built into a few products that are available today. Most software vendors have decided that they can satisfy us with razzle dazzle—pie charts that spin and bars shaped like pyramids—and so far we haven’t discouraged them by refusing to buy their silly products.

We are stuck in a vicious cycle of data impoverishment. Vendors show us bad examples of data visualization and we emulate them in our work. When vendors then look at what their customers are doing, they see examples that lead them to give us more of the same. Only a few vendors care enough for their customers to avoid the silly stuff that undermines our efforts.

This morning, I was faced with a fresh reminder of the current state of data impoverishment. A reader invited me to visit Microstrategy’s website to see the finalists of its customer dashboard competition. What I found was depressing. I couldn’t find a single example of a dashboard that could be used to monitor information effectively. At best they could be used to look up a few facts when what’s actually needed is a rich set of comparisons. The problems that I found are too many to delineate, but all the dashboards suffer from a common flaw: they say too little and what they do say they say poorly.

To give you a sense of what I found, here are three of the winning entries:

The U.S. Postal Service is currently struggling to adapt to changes in the way that people communicate. If they find this dashboard helpful, it’s no wonder they’re struggling.

This entire dashboard displays four measures, and none in a manner that’s particularly useful. The slight exception is that you can choose one of the four measures, such as Occupancy, which is currently selected above, to view a time-series display. Unfortunately, this combination bar and line graph performs poorly compared to a simple line graph with three lines.

Poor Eli Kiwanuka. It appears that he was photographed in a police lineup. This “Executive Dashboard Summary - Sales” displays the performance of one person only. How is this a summary of sales performance? Rather than a dashboard for monitoring performance, this provides a means to look up a few facts about individual sales people, one at a time, and requires that you wade through a series of poorly-designed, eye-assaulting graphs to painfully piece together a picture of one person’s performance.

I don’t blame Microstrategy’s customers. They’re working within the constraints of the tool and emulating impoverished examples, which is probably all they’ve ever seen. I mostly blame the folks at Microstrategy, who should know better. This competition gave the folks at Microstrategy a perfect opportunity to critique the designs that were submitted and show their customers how much better these dashboards could work if designed more effectively, assuming their software makes this possible. Did they miss this opportunity because they don’t know any better themselves?

Impoverished displays of data are what you get when vendors care a lot about sales but little about the real needs of their customers.

Take care,

8 Comments on “The Vicious Cycle of Data Impoverishment”


By Michael P. Manti. March 29th, 2010 at 7:04 pm

There were a couple customer dashboards that weren’t horrible. MySpace managed to get a reasonable set of sparklines, for example. But I agree with your overall assessment.

I’m not so sure that the vendors are wholly to blame. In my experience, non-quants often tune out as soon as you present them a data-rich display, even if it tells a compelling story that you carefully guide them through. I’ve had the most success when presenting numbers that confirmed the conventional wisdom.

Any ideas on how to sell displays that tell stories that the viewer has strong prejudices against?

By Jorge Camoes. March 30th, 2010 at 1:00 am

Tools are not neutral. While you can make a good chart in Xcelsius, that’s against it’s nature. It’s easier in Excel, but bad defaults are there. When people argue against Tufte’s “The Cognitive Style of Powerpoint” they fail to understand that.

Combine sales-oriented defaults, low graphicacy and bad examples and you get those charts and dashboards.

By Stephen Few. March 30th, 2010 at 9:15 am

Hi Michael,

Apparently the MySpace dashboard was not one of the winners. Although has as much merit as the others, it’s essentially just a set of tables of numbers with sparklines included. Everything in it follows a strict consistent formula without regard for differences in the data.

I haven’t observed that “non-quants” in general resist a data-rich display if it tells a compelling story about something that they understand, is well designed, and you show them how to read it. I suspect that people’s willingness to deal with necessary complexity in a data display is more a matter of their expertise in the domain than their mathematical sophistication.

By Stacey Barr. March 30th, 2010 at 10:27 pm

I wonder why people build dashboards. I mean, if they knew the specific questions they wanted to answer BEFORE they buy a dashboard application, maybe they’d have more awareness of when their dashboard failed to deliver.

In my line of work, managers and decision makers see dashboards as the solution to their performance monitoring and management woes. But the vast majority of these managers and decision makers have no idea what performance they need to monitor or manage, prior to investing gazillions of dollars into dashboard development.

I was having dinner with Gary Cokins and some of his colleagues at SAS Australia last night, and a good chunk of our conversation was around the importance of collaboration between the dashboard vendors and the consultants who can help clients build the performance management framework that the dashboard tools can help implement.

This is an important message to get out there (somehow): the dashboard won’t do the thinking for you, you need the performance framework (the business goals, the metrics, the targets) first.

By Larry Keller. March 31st, 2010 at 5:22 am

Michael,

The cultural adoption of good data visualization can be more readily accepted if the visual(s) are also rendered in cross-tab formats. Further,I consider data visualization like a story-board as used in advertising. The author/publisher should create what he/she knows to be of audience value in the most simplistic yet flexible manner. There will always be naysayers but once convinced, they become advocates

By Michael P. Manti. April 1st, 2010 at 3:53 pm

By “non-quants”, I meant people who aren’t accustomed to looking at or thinking about data, regardless of mathematical or statistical sophistication.

I regard a compelling story, domain expertise, good design, and helpful instruction to be necessary but insufficient conditions for engagement with a data display. I think a willingness to allow evidence to influence one’s opinion is also a prerequisite. But if someone’s too tired, too preoccupied, too vested in the status quo, etc., she’s much less likely to put forth the required effort, even if she’d be otherwise willing.

By Chris Holtom. April 7th, 2010 at 9:45 am

Stephen, thank you for saying what the majority seems not to have noticed. I thought a dashboard was designed to draw your attention to anomalies or, at a glance to tell you that things are OK (pilot’s view) I always remember the Commander of a US Army Corps whose staff spent their time maintaining a dashboard of coloured states (artillery shells, serviceablwe helicopters etc) He was only interested in the ones coloured red, because the staff knew what parameters of risk any plan could bear. In this way the endless briefings and exchange of unneccessary information which blights large organisations (notably the military-surprisingly) became unneccessary.

If a dashboard does this, by accumulating red (risks) upwards to decision makers then it has substantial value I guess. Stacey Barr says this rather more elequently above.

By Jane Maruszewski. April 20th, 2010 at 7:47 am

Stephen, we’re glad you’re taking a look at some of the ways in which MicroStrategy’s dynamic dashboards are employed. Aesthetics can be subjective. In this case, the dashboard you’ve critiqued was designed within hours and resulted in $2M cost savings within the first 3 months of deployment. That’s the larger story here. As for the photo, we enjoyed your commentary – this is a shot of one of our veteran employees, which I’m sure he would rather bury… but some data and specifics were changed for customer privacy reasons.