Dental work by road workers with jackhammers — Dashboard design gone awry

Alright, I’m not really going to write about the having your dental work done by road workers with construction equipment, but I am going to write about something that is just as painful and absurd: information displays that are designed by software engineers who know nothing about design.

A few days ago a press release was published by the George S. May International Company to announce its dashboard solutions for “small to mid-sized” companies. Here’s a quote from the press release:

We have found that two major obstacles stand in the way of business owners managing their companies more effectively. One is the difficulty in understanding the data they have. The second is difficulty in determining the cause-and-effect relationships among the different data. Management Dashboards helps business owners overcome these obstacles.

While this accurately describes two common problems in business today, I don’t agree that the dashboards that George S. May offers do much to solve the problem. Like most dashboard providers, this company’s solutions communicate information poorly. Effective dashboards result from a combination of good technology and good design. These dashboards look like they’ve been designed by technologists who sit in their dimly lit cubicles all day banging out code, isolated from the world of people. Dashboards are a medium of communication. To work effectively, they must be designed to present the information that people need to do their jobs in a way that is clear, accurate, and efficient.

It makes me sad and even a little angry when software and service companies advertise information solutions that work this poorly, because it isn’t that difficult to learn how to do this right. To illustrate this point, I asked Bryan Pierce, the Operations Manager at Perceptual Edge who has been working with me since last December, to critique one of the dashboards that are featured by George S. May. Before coming to Perceptual Edge, Bryan had no experience with data visualization, and because his work doesn’t require him to be an expert in this field, what he has learned he has picked up mostly indirectly, by reviewing my articles, books, and blogs. In a short time, he has developed the skills that a company such as George S. May could use to produce solutions that really work. The rest of this blog entry was written by Bryan to illustrate how easily the visual design skills that are needed to dramatically improve dashboards can be learned.

Take care,

Signature

 


My name is Bryan Pierce. I am not a rocket scientist or a brain surgeon, nor do I need to be to understand and apply the principles of good information dashboard design. For the last six months, I have worked with Stephen Few at Perceptual Edge, handling the day-to-day operations. Using the skills I have picked up in that time, I am critiquing and providing recommendations for the improvement of a dashboard I recently found online, which was created by the George S. May International Company (http://www.gsmdashboards.com/): 

Prior to working with Stephen, I had no exposure to information dashboards; I wasn’t even familiar with the term. Just after Stephen offered me this job, I decided to read Information Dashboard Design so that I’d have a better understanding of Stephen’s work. Over the past few months, I’ve also read most of his articles and blog posts that address the subject. With the exception of a few conversations we’ve had on the subject, everything I know about effective dashboard design can be learned from Information Dashboard Design or http://www.perceptualedge.com/.

My discussion of this dashboard’s problems is broken into sections. First, I’ll discuss the overall problems, and then I’ll point out some of the problems that are specific to individual components.

Overall Problems:

  • Layout: Each component on the dashboard fits into an equally sized “box,” which scales when the window is resized. All of the components resize along with their containing boxes, except the table, which does not scale. Depending on your resolution, this can cause the dashboard to be unbalanced, as in the screenshot above, where some of the items are unnecessarily large, while the table is almost illegibly small. Besides poor scaling, the dashboard’s layout is hindered by the fact that it is based on a grid. As mentioned before, each component fits into an equally sized box, even though all components probably shouldn’t be equally sized. For instance, the heatmap (bottom left) has a much higher data density than the upper bar graph and should probably be allowed more space, yet the dashboard’s grid system gives them the same amount of “real estate.” More thought should have been given to the size and placement of each of these components, based on the nature of the data and its intended use.
  • Fill Color: Fill color is used to separate the dashboard into four sections. This makes it unnecessarily difficult for your eyes to track between the differently colored columns. In this case, white space alone would have probably been enough to delineate the sections (with a proper layout); if not, very thin, light gray lines could have been used. In the instances where it is necessary to use fill color to separate sections, a very light color is all that is needed.
  • Contrast: The contrast of the graphs compared to the background colors varies significantly. For instance, the heatmap and the gauge use bright colors on a dark background, so they are the most visually salient objects on the entire dashboard. But, are they really that much more important than everything else? Now look at the table and the upper bar graph. They both use blues that are very similar to the background color. As such, they fall away into the background. While a good design can use differences in contrast to direct our eyes, in this case, I think these differences are arbitrary.
  • Lack of Context: Many of the graphs are hard to decipher due to insufficient explanatory text. For instance, the “Average Loan Size” bar graph would be easier to understand if it said what units it was being measured in (e.g. U.S. dollars, thousands of U.S. Dollars, etc.). In some cases, such as in the pie charts, the missing information can be obtained from a pop-up legend, by clicking the small eyeball icon to the bottom-right of the charts. However, many of these graphs could have easily been put in context through clearer titles and labels, making the pop-up legend unnecessary. Also, even with the assistance of the legend, some of the graphs are still indecipherable. For instance, notice that both the gauge and the table display the “loan count,” but represent drastically different values. In context, it’s likely that both of these values would make sense. Unfortunately, that context has not been provided.
  • Vertically-Oriented and Angled Text: The line graph, the two bar graphs, and the Pareto chart (top right) use vertically-oriented text for their axis titles. The bar graphs and the Pareto chart also use angled text for some of their labels. Vertically-oriented or angled text is harder to read than horizontally-oriented text and should not be used if it can be avoided. On this dashboard, the vertical axis titles could easily be moved to the top of the axis and horizontally-oriented, while the labels could all be oriented horizontally without moving them at all (although some of the labels in the Pareto chart would need to be split onto two lines).
  • Unnecessary Precision: Graphs are used to show the shape of data, to compare magnitudes, spot exceptions, etc. If exact values are necessary, a table works best. As such, it’s usually not necessary to show actual values of bars; a scale along the axis will likely provide sufficient precision. In this dashboard, the numbers have been written directly on the bars and pie slices, and in many cases they have been written to two decimal places of precision. This clutters the graphs and distracts us from the shape of the data. On the rare occasions when the exact values and the shape or magnitude of the values are both necessary, a table and graph should be used in conjunction. It’s less distracting and more efficient to look up values on a table that is below or next to the graph than it is when they’re integrated.
  • Use of Color Gradients: It’s a rare occasion when the use of a gradient in a dashboard actually serves to enhance its usability. Most often, color gradients are used in a misguided attempt to make a dashboard more visually interesting. At best, this is useless decoration; at its worst (such as when a gradient is used in the plot area of a graph), it can actually cause optical illusions that can adversely affect perception of the data. In this dashboard, gradients are used to decorate many of the graph and axis titles. This does nothing to enhance communication and only serves to give these titles unnecessary visual salience.

The Heatmap:

  • A heatmap is a poor choice for the display of time-series data. In addition to the actual values, which a heatmap can only display in a very rudimentary manner, based on color, it’s often useful to see the shape of change through time. If a line graph were used instead of a heatmap, it would be much more enlightening. For instance, in June, the heatmap shows us that for all but one day, the amount of loans funded was considered “Poor.” If a line had been used, we could see whether the amount of funded loans is trending upwards, downwards, or remaining flat, whether the loan amount is fluctuating significantly between days or remaining fairly steady, etc. Reference lines could be used to signify the division between “Good,” “Fair,” and “Poor” performance.
  • The horizontal axis of the heatmap is inadequately labeled and very confusing. The axis label says that each number represents the “Date,” but last time I checked, February had more than 19 days. After working at it, I was able to decipher the meaning of the days. Each number represents a business day for a given month in the year 2005. In addition to ignoring the weekends, the heatmap also ignores certain holidays. For instance, February only had 19 work days if you exclude President’s Day. As you can see, the problem with this is that nobody thinks in terms of work days. You don’t think, “Today is the 13th work day of the month.” You think “Today is the 17th day of the month.” The heatmap’s design would have been much more effective if every day of the month was included and non-business days were simply left blank or “grayed out” in some manner.
  • Color is used poorly in the heatmap. The use of similar intensity reds and greens together makes the heatmap useless to the 10% of men and 1% of women who are colorblind. Additionally, by only encoding three different values (“Good,” “Fair,” and “Poor”) we lose out on some of the depth the heatmap could have provided. For instance, currently, a single loan could mean the difference between a day being considered Fair or Good. If a divergent color scale were used—that is, one that uses different intensities of two different colors, to encode the data—the heatmap would provide much more insight. For instance, red could be used for poor loan days and blue could be used for good loan days. Days with extremely high or low loan volumes would show up as bright red or blue, average days would appear gray, and everything else would fall somewhere in between. While we still wouldn’t know exact values (color doesn’t work for this), we would have a better idea of just how “good,” “poor,” or even “fair,” each day was.

The Gauge:

  • The primary problem with the gauge in the second column is, well, that it’s a gauge. Gauges are “all the rage” on information dashboards; unfortunately, they take the dashboard metaphor too far. One of the strengths of gauges on real automobile dashboards is that they are an easy way for a mechanical device to show change through motion. For the needle to change, its base only needs to rotate, instead of physically moving from one place to another. However, computerized dashboards need not share the same physical restraints that real world gauges do. On a computer screen, the circular shape of the gauge only wastes space and makes it unnecessarily difficult to read. Additionally, it’s rare that the data on a dashboard is updated so frequently that motion will actually be used to show change. This dashboard is no exception. The “real-time” gauge represents the “loan count” for a given month. You would want to know this number, but would you really watch the gauge to see how fast it changed, the way you look at the speedometer in your car? No. The information contained in the gauge could be displayed more efficiently in a variety of ways that would also save room. One of the best ways to display this information is through the use of a bullet graph, which Stephen developed specifically as an effective, compact replacement for gauges.

The Upper Bar Graph:

  • Can you name the two months that follow May? June and July, right? When you think of them, you never think “July and June,” because that is not their order in the year. Anytime time-series data is shown on a graph, it should always be sorted from earliest to latest and never any other way. Unfortunately, in an attempt to rank the months by “Average Loan Size,” the creators of this dashboard put July before June, as seen in the upper bar graph. They shouldn’t have.

The Pareto Chart:

  • The graph in the top right corner is called a Pareto chart. In this type of graph, the bars indicate the magnitude of individual items, while the line indicates the cumulative totals of those items from left to right. For instance, the line starts at the top of the first bar and then as we move to the second bar, the line displays the combined total of the first and second bars. Once we reach the right-most bar, the line equals the total of all items. The problem here is not the Pareto chart itself, but the scale on the left. The scale on the right expresses each bar’s ACH Pull as a percentage of the whole (which is why the line ends at 100% on this scale), but it is not clear what the left scale represents. Also, the unequal precision used on the left scale, with numbers containing anywhere from 0 to 2 decimal places, makes it more difficult to read than necessary.

The Pie Charts:

  • Pie charts should never be used. Research has found that people have a much harder time accurately judging 2-D areas, such as slices of a pie, than they have comparing lengths, such as the lengths of bars. These pie charts also exhibit another problem that most other pie charts do not. Because, both dollar values and percentages are provided on the pies, it’s natural to assume that comparisons can be made between the slices of two different pies. However, this can only be done with the percentages, not with the dollar values. For instance, look at the blue portion of the pies that represent February and March. Although February’s blue slice is larger and represents a larger percentage of that month’s total, the dollar value that it represents is significantly less than the dollar value for March. This problem and the problem of pie charts in general could be avoided if this was redesigned as a bar chart. Each pie could be replaced with a pair of bars, and all of the bars could be placed on a single axis. The vertical axis scale could be in dollars. This design would be more compact than six pie charts, and it would make comparisons between the magnitudes of two bars in a single month or between two bars in different months accurate and efficient. If the exact percentages were necessary—which they probably wouldn’t be, given the higher accuracy of magnitude comparisons with bar graphs—they could be included in a small table below the graph.

There are other tweaks and polishes that could be made to the dashboard; I have only discussed the most egregious problems. So, why do so many companies make dashboards that have so many design problems? I don’t know. Maybe they are unaware of the problems or perhaps they just don’t care. I can tell you, however, that it’s not difficult to learn the principles of good dashboard design. I have picked them up through a few hours of reading. Once they’re explained to you, they make sense and eventually seem intuitive. It only takes a little effort to learn how to create dashboards that are as effective as they should be.

9 Comments on “Dental work by road workers with jackhammers — Dashboard design gone awry”


By Scott Taylor. May 17th, 2007 at 1:57 pm

Great write up on a product that likes to use the crayon approach. Use every thing in the box no matter if it makes sense. I really like the link to the bullet graph discussion and ultimately to the Excel User tutorial. Thanks for this information. In the mean time I’ll try to duplicate the bullet chart in Cognos 8 Report Studio, wish me luck.

By Jochen. May 20th, 2007 at 11:35 pm

> Pie charts should never be used.
Hi Stephen & Bryan,
We use pie charts in the MDG Dashboard, and we have good reasons to do so. Your arguments against pie charts are known and valid, but hammering the message in this apodictic way puts you at risk to be regarded as “ideologic” - which would be a pity ;-)

By Stephen Few. May 21st, 2007 at 10:29 am

Jochen,

The ineffectiveness of pie charts, although obvious to you, is not obvious to most people, which is why I point this out from time to time. If I were expressing my opinion only, rather than basing my objections to pie charts on empirical evidence, my objections might appear “idealogic”, but I’m careful to base what I say on firm evidence. I’m battling ineffective practices that run contrary to the evidence of what works. This effort warrants a bit of passion.

If you are familiar with the problems that are associated with pie charts, I would be interested in seeing an example of their use that warrants their use in the MDG Dashboard. Perhaps you can post an example in the Discussion Forum at Perceptual Edge as a means to generate some discussion.

By The Dashboard Spy. June 11th, 2007 at 6:11 am

As a third party with many years of experience in moderating debates about user interface design in general and dashboard design in particular, let me interject a few points before this becomes merely a “because I said so” type of debate.

Let’s decide on the criteria for “good design” on a dashboard. Yes, you can each argue about the clarity of the information or the applicability of the pie chart, but I think there is a person whose opinion counts more than any of us - that being the end user.

The end user is the ultimate arbiter of whether a dashboard is “good” or “bad”. Yes, that involves the data being clearly portrayed in a way that is free of bias, but it also transcends the data visualization elements and encompasses the design of the overall application (the usual usability concerns regarding the application’s information architecture, navigation, look and feel, etc).

The end user’s reaction can truly be determined after the fact with testing, but can certainly be anticipated with a solid grasp and application of the current best practices in dashboard design. Mock up the suggested design with a high-fidelity example and float it with the user community before insisting upon it.

It is through the study of dashboard design examples that we determine what works and what doesn’t. This is why I have collected over a thousand screenshots of information dashboards at enterprise-dashboard.com.

The best practices become clear in terms of “what not to do” when you study dashboard after dashboard.

What we need to watch out for is when vendors go with certain data visualization design solutions because the current version of their software only allows a certain type of chart or analysis.

I know that vendors live in the real world of feature set contraints - I accept that. I do ask, however, that they adopt an attitude that works with the folks who maintain a “best practices” approach. The vendors can acknowledge the wisdom of the suggestions and say that they will address the situation by recommending the solution as a future upgrade.

By Stephen Few. June 11th, 2007 at 6:38 am

Mr. Dashboard Spy,

The criteria that I use for determining the effectivness of any form of communication is whether people understand it — how thoroughly, clearly, quickly, and accurately they understand it and can respond to it in a meaningful way. The visual design or widget preference of the end user often does not correspond to the effectiveness of the communication. Should a doctor allow the patient to tell her how to perform the surgery? Not if the patient wants to walk away afterwards with his health intact. A great deal of research has demonstrated that people often prefer products that work poorly over those that work well, simply because of things such as flashy visual design. Unfortunately, most of the dashboard examples that vendors produce and most of those in your collection are poorly designed. They feature visual presentation that is dysfunctional, but when end users only see these poor but flashy designs, they assume that this is what qualifies as the way it ought to be.

Simply looking at many examples of dashboards does not hone one’s ability to recognize what works and what doesn’t. Only if you critique them, based on solid evidence from research about the effectiveness of various visual designs and mechanisms of display, can you learn from the examples.

The final arbiter of an effective visual design is not the end user, who has no training in visual communication, but a designer who is well-versed in the research. End users know the business they are using the dashboard to monitor, but they rarely are trained in visual design.

Please don’t be concerned with any discussion on this site degenerating into an endless back and forth of conflicting opinion. When authority is given to the evidence, opinions must fall in line or  remain silent. This is a responsibility that I take seriously.

Take care,

Steve

By The Dashboard Spy. June 11th, 2007 at 8:33 am

Steve - thanks as always by your clarity in thinking. I finally found someone who gets the real message behind the 1000’s of dashboard screenshots that I’ve posted - THEY ARE ALMOST ALWAYS POORLY DESIGNED!!! In my opinion, there are less than 10 screenshots in my collection that stand out as examples of truly great design.

There is a sad level of design skills right now in this field. This is being slowly improved but there is much more ground to cover. Of course, you know this, but let me bring to the attention of the dashboard community that there is only one book out there on the best practices of designing information dashboards - Information Dashboard Design: The Effective Visual Communication of Data by Stephen Few (yes folks, the very same Mr. Few! - seriously, this book is an absolute must read that you should order today.)

My point in bringing up the end user, however, is that without the buy-in from them, we run the risk of “losing” them and having them think of us as ivory tower academics rather than fellow business partners.

And certainly, I’ll restate that dashboard designers should not “make do” by the limitations of the software vendors involved. This is one of the prevalent reasons why so many dashboards in my collection miss the mark.

Steve - can you have Bryan Pierce rejoin this conversation as well? I think that your readers would love to hear his input on this as well. I know I would. Thanks, Dashboard Spy

By Bryan Pierce. June 12th, 2007 at 9:39 pm

The Dashboard Spy,

I appreciate that you’d like my input in this discussion, although I doubt that I could add any new insights that Stephen hasn’t already discussed somewhere on this blog.

In my capacity as Stephen’s blog administrator I read every entry and comment that is posted to this blog. As such, I’ll keep my eyes open, just in case I need to jump in and help Stephen out.

-Bryan

By Joe Sixpack. June 14th, 2007 at 3:20 pm

I’ve never seen a dashboard as good as this one. The background colors make it easy for the user to differentiate the sections, and the Heat Map is the best that I’ve ever had the honor to gaze upon.

-Joe

By Jon Peltier. June 16th, 2007 at 6:47 am

Mr. Dashboard Spy -

I used to subscribe to your dashboard blog. After some time, however, I stopped. Almost all of the entries seemed to be awful, and your lack of critique implied that you endorsed them as not just representative examples, but also as exemplary samples of effective dashboard design.

If you would spend half an hour commenting on each dashboard you post, and perhaps mocking up a more effective substitute, it would make your blog and your collection of dashboards a much more worthwhile read,