In the recent Jan/Feb/Mar 2013 edition of the Visual Business Intelligence Newsletter, I introduced an extension of Tufte’s sparklines called bandlines. In less than two weeks since their introduction, without any prior knowledge of them, the software company XL Cubed has incorporated bandlines into their product. Here’s a screen shot of an example:

XL Cubed works as a companion to Excel. It can be used to extend Excel’s data visualization and analysis capabilities, including the addition of bullet graphs. You can read about this implementation in their recent blog post appropriately titled Bandlines in XL Cubed. This is not an endorsement of the product (I don’t give endorsements), but merely an announcement that one software company so far has recognized the value of bandlines and beat all others out of the gate.

Take care,

We’re all familiar with Edmund Burke’s well-known line “Those that don’t know history are destined to repeat it.” Burke’s insight applies in spades to the history of data visualization and its practitioners. Each new wave of practitioners insists on rushing in and claiming the laurels of expertise without first studying the lessons and mistakes of the past. We’d be much further along today if this weren’t the case.

I recently read the book Practical Charting Techniques by Mary Spear, which was published in 1969. This was several years before books on graphics by John Tukey, Edward Tufte, or William Cleveland. Spear’s early work is full of practical wisdom and it focuses on best practices as she understood them at the time. Many of the visualizations that I introduce to students in my courses today, which they find new and exciting, can be seen in the pages of this book. Spear included a graph that looks and works almost exactly like Tukey’s box-and-whisker plot years before he introduced it. She even includes graphics that look like an early predecessor of my bullet graph. What I found particularly interesting are two statements that she made about the state of data visualization back in the 1960s.

Here’s the first:

In the mid-1940s and 1950s the three-dimensional chart was popular. Fortunately that vogue seems to be passing. While some were interesting and attractive, they were for the most part practically impossible to read or interpret directly. Comparisons were difficult to make, and scales had to be visually adjusted so that amounts could be read. Such a process required mental and optic gymnastics. (p. 62)

She bemoaned the popularity of 3-D graphs and explained why they fail, but was encouraged by their decreasing popularity. What happened between the decline that she witnessed in the 1960s and the renewed popularity of 3-D graphs today? Software companies got involved in creating tools for the creation of graphs without taking time to learn the lessons of the past. No companies are more to blame than those in the business intelligence industry. Had they built their tools responsibly, we could have avoided another dark ages of graphical display.

Here’s the second of Spear’s statements that I found particularly interesting:

Statistics can be as misleading as the intentionally distorted chart is. Surveys and samples can be biased, correlations too small, factors missing, improper measures taken—not to mention the possible prejudice of the writer or speaker. But fortunately, as the art of graphic presentation advances, more listeners and viewers are becoming aware of these pitfalls and give heed to them; and more designers of charts are being artistic without confounding the true story. (p. 68)

Advancements in data visualization were already, way back then, leading people from a juvenile proclivity to decorate graphs in ways that undermined their stories into a more mature approach to graphical communication. At some point, however, the progress that Spear observed and appreciated was waylaid by a return to silly graphics. Much of this corresponds to the rising interest in infographics, which encouraged students of the graphic arts to venture into the realm of data visualization without bothering to first learn from the past. Graphic design skills can indeed be applied to data visualization, but not without first learning about the field and how it differs in purpose from other uses of graphics. While a few thoughtful infographic designers have honed their craft by studying data visualization best practices and the realms of science that inform these practices, most have taken the fast track born of arrogance and built on ignorance. Infotainment is the result: displays that entertain with flashy tricks while the information on which they’re based becomes unrecognizable and distorted in the glare.

Must every new generation repeat the mistakes of the past? Can’t we use technology for advancement rather than as a streamlined path to dysfunction? Only if we review our history and learn from it.

Take care,

I have finally created my own version of the Student Performance Dashboard that contestants in the 2012 Perceptual Edge Dashboard Design competition were asked to create. I don’t feel that I should judge the efforts of others unless I’m willing to submit my own work for scrutiny as well. This version along with those created by the two winners will appear in the second edition of Information Dashboard Design.

(Click to enlarge.)

Examine this dashboard on your own for a few minutes. Before reading further, examine each measure and the way it’s expressed, including the context. Look at each component, both on its own and in relation to the whole. Consider the overall visual design: how it draws you into the information and draws your eyes to what’s important.

Hopefully, the reasons for each of my design choices became clear as you examined it closely. You may have noticed that I incorporated several of the ideas that were exhibited by dashboards that were submitted to the competition, especially the two winning solutions. Yes, I cheated, and for this reason I didn’t give myself an award. Here are a few of the good qualities of this dashboard that were present in others as well:

• All of the information is present.
• It is easy to spot the students who are most in need of attention.
• The organization is clear.
• The students that most need attention are clearly featured, using simple blue icons.
• Graphics have been used to support efficient scanning of the information.
• Everything about a student can be seen by scanning across a single row.
• Students can be easily compared by scanning down the columns.
• Even though there is a great deal of information, little training would be required to learn how to interpret this dashboard.
• The information has been displayed in an aesthetically pleasing manner.
• It is scalable in that more or fewer students could be accommodated by simply adding or removing rows.

Now let’s consider a few ways that this design succeeds where others fell short.

• Student-level and class-level information has been well integrated.
• The sparklines are more informative.
• It is easier to see time-based attendance patterns (absences and tardies).

By placing class-level summaries below related student-level information, the relationship between them is clearly shown and comparisons can be easily made.

The sparklines are a variation of a version of Edward Tufte’s space-efficient invention, which I call bandlines, that I introduced in the current edition of the Visual Business Intelligence Newsletter in an article titled, “Introducing Bandlines: Sparklines Enriched with Information about Magnitude and Distribution.” In this case, I’ve used horizontal bands of color to represent ranges of scores that correspond to grades A, B, C, D, and F. With this design, I was able to provide the teacher with a quick glimpse of historical student achievement that reveals not only patterns and trends but also information about the magnitudes and variability of values. Usually, bandlines use bands of color to represent information about how a measure is quantitatively distributed based on quartiles, similar to a box plot. As such, it is adaptable to a broad range of measures.

To show historical attendance information, I designed a display that was similar to the winning solution by Jason Lockwood, but is a little easier to perceive and comprehend at a glance.

I hope you can appreciate the design choices that I made to produce this dashboard and understand how they support performance monitoring. I have no illusion that this version of the Student Performance Dashboard is perfect. I have never designed anything that I couldn’t improve later. As new ideas come to mind, several of which will no doubt come from you, I’ll continue to improve this dashboard with each new printing of the book. Despite the evolutionary nature of design—time is a great teacher—I’m confident that this dashboard could be used by teachers to help their students achieve their best.

Take care,

In 1986 Carlo Petrini founded the Slow Food Movement. This movement began as resistance to the opening of a McDonald’s fast food restaurant near the Spanish Steps in Rome. I’ve seen this affront to the old city and felt the disgust that must have emboldened Petrini to start an international movement. Slow Food was introduced as an alternative to fast food. It is based on the belief that much of the beauty and wholesomeness of food requires that we take time with it: time in producing it, time in preparing it, time in savoring it. The Slow Food Movement is one of a broader Slow Movement that focuses on many aspects of life. I learned only a few days ago that there is a Slow Reading movement that encourages people to slow down and appreciate what they read. It is not surprising that in our fast-paced world it is important to be reminded to slow down and embrace life with greater awareness and appreciation, lest we forget who we are and what makes life worth living. I believe that it is time to extend the Slow Movement to the realm of information technology. In this time of so-called Big Data, too much is being missed in our rush to expand. The entire point of collecting data—using information to better understand our world and then make more informed decisions based on that understanding—has been forgotten and is certainly not being achieved in our manic rush to throw more technology at a problem that can only be solved by making better use of our brains.

In the last few days you have no doubt read many predictions about the new year. I am loathe to make predictions. I find that most predictions made about technology fall into one of three categories: 1) statements of the obvious (e.g., people will increasingly rely on tablet devices), 2) marketing (e.g., our product will lead the way), and 3) outright guesses (e.g., Gartner’s recent prediction that by 2015 a total of 4.4 million jobs will be created worldwide to support Big Data). Rather than making predictions as a way to start the new year, I’d rather state my hopes. A rising appreciation for Slow Data and the practices that naturally arise from it is my hope for this year.

Big Data is usually defined in terms of the 3Vs: volume, velocity, and variety. Doug Laney of Gartner originally defined the 3Vs 12 years ago. When he wrote his original paper on the topic, the 3Vs were already old news. I remember reading Laney’s paper at the time and thinking that he did a good job of characterizing significant aspects of data that have been true since the advent of the computer. Actually, if we want to be historically accurate, we can date the beginning of Big Data to the year 1440 when Gutenberg invented the printing press. I believe that the advent of the printing press had a greater impact on the world of information in terms of volume, velocity, and variety than the advent of computers. Actually, long before the printing press the invention of writing had an even greater impact and before that the invention of language even greater. What’s happening with data today has its roots firmly planted in a long line of technologies that have allowed humans to disseminate information for ages. Technology increases data volume, velocity, and variety. The fact that these have increased at an exponential rate since the advent of the computer is well known and has been for years, yet packaged as Big Data and fueled by huge marketing budgets, this growth is suddenly being embraced as something new and unprecedented. Hurray data! Hurray technology! Three cheers for the technology vendors that are making a bundle selling incremental extensions of what they’ve been selling all along. While the world reaches for its wallet amidst the rising clamor, what’s important about data is being lost in the din.

I’d like to introduce a set of goals that should sit alongside the 3Vs to keep us on course as we struggle to enter the information age—an era that remains elusive. May I present to you the 3Ss: small, slow, and sure.

Small

As data increases in volume, we should keep in mind that only a relatively small amount is useful. Data consists of a lot of noise and only a little signal. We must separate the signals from the noise, which we’ll never get around to doing if we spend all of our time boosting technology for data generation and collection, but not learning how to find and understand what’s actually meaningful and useful.

Slow

We’re in love with speed. Like many people, I love to drive fast. It’s a rush. Much of what I value in life, however, requires time. This is especially true of data sense-making and decision-making. Some of my favorite words were spoken by Lao Tzu, the founder of Taoism:

Muddy water, let stand, becomes clear.

These words have come to mind and thus to the rescue many times in my life. I recently read a new book titled Wait: The Art and Science of Delay by Frank Partnoy, which roots the benefits of waiting, pausing, taking a bit more time, in science. In the introduction Partnoy says:

The essence of my case is this: given the fast pace of modern life, most of us tend to react too quickly. We don’t, or can’t, take enough time to think about the increasingly complex timing challenges we face. Technology surrounds us, speeding us up. We feel its crush every day, both at work and at home. Yet the best time managers are comfortable pausing for as long as necessary before they act, even in the face of the most pressing decisions. Some seem to slow down time. For good decision-makers, time is more flexible than a metronome or atomic clock…As we will see over and over, in most situations we should take more time than we do.

Although some decisions in life are best made instantly based on intuition, this is only true if your intuitions were built on a great deal of relevant experience and the matter at hand does not lend itself to deliberation, such as a bear running towards you at full speed. These are the types of decisions that Malcolm Gladwell wrote about in Blink. Most non-routine decisions, especially those that change the courses of our lives, benefit from conscious, deliberate, analytical reasoning—what psychologists such as Daniel Kahneman call “System 2 Thinking.” In fact, Kahneman refers to these two modes of reasoning as thinking fast and slow.

No matter how fast data is generated and transmitted, the act of data sense-making, which must precede its use, is necessarily a slow process. We must take time to understand information and act upon it wisely. Speed will in most cases lead to mistakes. Bear in mind the wise parable of the tortoise and the hare.

Sure

Even though we can collect data about everything imaginable, variety is not always a boon. More choices are only helpful if 1) we need them, and 2) we have the time and means to consider them. Otherwise, they do nothing but complicate our already overly complicated lives. In an effort to remain sane, I spend a fair amount of time limiting my choices. For instance, I don’t participate in Twitter, text messages, Facebook, or even the professional social networking service Linked-In, because I already face enough interaction with people as it is. By restricting myself mostly to email correspondence and direct face-to-face conversations, I maintain the level of human interaction that works for me. I’m not suggesting that these services are bad, but they don’t suit me. The next time that you’re in a grocery store browsing the toothpaste section, ask yourself if the variety of products arranged in daunting rows is useful. Wouldn’t just a few good choices make life better?

Life and our world are rich in variety. This is a good thing. Data consists of a collection of facts about life and the world. Only a subset of those facts will be useful to you. The same is true for an organization. Just because you can collect data about something doesn’t mean you should. In fact, given all the data that you’ve already collected, wouldn’t it make sense to spend more time making use of it rather than getting wrapped up in the acquisition of more? When you recognize an opportunity to do something useful with data, that’s when it becomes sure. As people and organizations of limited resources, shouldn’t we spend our time identifying what’s useful and then actually using it?

Data is growing in volume, as it always has, but only a small amount of it is useful. Data is being generating and transmitted at an increasing velocity, but the race is not necessarily for the swift; slow and steady will win the information race. Data is branching out in ever-greater variety, but only a few of these new choices are sure. Small, slow, and sure should be our focus if we want to use data more effectively to create a better world. I doubt that the 3Ss will ever become the rallying cry of a mighty movement, but those who heed them will become the true heroes of the information age. When the dust settles, we’ll see that it was people who took the time with a limited collection of the right data who solved the problems of our age.

Take care,

Have you noticed that many business intelligence (BI) software companies have introduced black screens as the standard for mobile devices? Is this because mobile devices work better with black screens? If you look for the research, as I have, it isn’t likely that you’ll find any. Just as when dashboards were new and someone came up with the bright idea of using graphs that looked like speedometers and fuel gauges on cars and everyone else followed suit, the practice of black screens on mobile devices has been adopted for the same reason: someone did it and others followed.

Few software vendors in the BI space do research or even read relevant research by others. A software developer does something on a whim and others copy it. Vendors find this approach faster and cheaper than research. The results, however, are costly to those who use the software.

I recently gave a keynote presentation at one of Actuate’s events and also led a separate smaller session to discuss data visualization best practices. During the smaller session, the issue of black screens on mobile devices arose—a practice that Actuate emulates. A fellow from Actuate made a passionate case for black screens by taking a bright flashlight, shining it directly into his face, and proclaiming, “This is what happens when you use a white screen on a mobile device.” His point was dramatic, but erroneous. The amount of light that is emitted from mobile devices with white screens is no greater than that emitted by laptop or desktop screens. It is not like staring into blazing light.

The same vendors that advocate black screens for mobile devices when graphics are involved, such as in a dashboard display, conversely advocate white screens for applications that involve reading, such as e-books. Illustrated below, RoamBI uses a black background for its analytics applications and a white background for its reports application.

No one seems to question the efficacy of light backgrounds for reading text. Why the difference? Text and graphics both involve objects that are constructed of lines and filled in areas of color. Do they differ in a way that demands a different background color? I don’t think so.

If you’ve worked with computers as long as I have, you probably remember that the original CRT displays used black backgrounds (i.e., the absence of light) and projected green phosphor pixels to form text. I remember the first time that I saw a screen that didn’t render text in green but did so in orange, which was a delight. Those old screens were hard on the eyes. As display technologies improved, light backgrounds became the norm for most applications. As it turns out, there was a reason for this. White or slightly off-white screens provide a better background against which information, whether in the form of text or graphics, stands out clearly and is easy on the eyes.

When I’ve asked vendors and others about the emergence of black screens on mobile devices, I’ve encountered the following arguments for their use:

1. Black is the absence of light, so a screen with a black background uses less energy, which extends the time that a mobile device can run without having to recharge the battery
2. Mobile devices are used in a broader range of lighting conditions, including direct sunlight, and black screens are less reflective than white and thus easier to read in direct light
3. White screens emit too much light and are therefore hard on the eyes (the explanation provided by the fellow from Actuate)
4. Bright colors on black screens look cool
5. Everyone else is doing this, so there must be a reason

If we’re interested in effectiveness—the users ability to see information as clearly as possible—the last three arguments above can be dismissed without further consideration. The first argument, regarding the preservation of the battery’s charge, sounds plausible, but it might be based on a misunderstanding of the display technology. When I have questions about uses of color in data visualization or about display technologies, I usually consult my friend and colleague Maureen Stone, the author of A Field Guide to Digital Color and long-time researcher in the field of information visualization. When I asked Maureen about the battery life issue, she said the following:

On an LCD display, the pixels fundamentally act as shutters for the backlight, which is on all the time whether the screen is black or white. If you want to save power, turn down the brightness.

Even if some forms of mobile displays operate in a way that preserves battery life when the screen is black, does it make sense to sacrifice usability for the sake of a little extra time between charges? Clearly, makers of e-book readers wouldn’t consider this a worthwhile compromise.

This leaves us with argument number 2 above regarding lighting conditions. To test this informally, I took my new iPad outdoors to view it in different lighting conditions, including direct sunlight, and didn’t find that a black background was less annoyingly reflective. In fact, the opposite appeared to be true. The black background acts like a mirror, providing a surface that is reflective enough for checking my teeth for stray flecks of green after a meal. I didn’t want to rely on informal observation alone, however, so I consulted Maureen about this. Here’s her response:

The glass is reflecting the ambient light. It doesn’t matter what the screen is emitting. The value that reaches your eyes is the sum of the two. The variation in the reflected light is much more obvious and distracting on a black screen than a light one because the contrast between the images created by the reflected light and the background is less when the background is white.

What Maureen described in the last sentence is the mirror-effect that I noticed when I informally observed the glare on my iPad.

As an expert in this field, Maureen shared a few general thoughts on the topic as well:

In general, I recommend a light or white background because that gives your visual system a constant place to focus and also controls your white adaptation. A fundamental part of color vision is perceiving colors relative to the current “white.” If you have colored text floating on a black background, it can feel less stable and in theory, you can get some differences in color perception as your adaptation shifts.  You can also create focus problems: An extreme case is intense red letters on black vs. intense blue ones on black. You can’t really get both in focus at once because of chromatic aberration in the eye’s optical system. This is easy to create on a CRT, less so on an LCD.

Another way to describe it is that a white background gives you more a feel of a constantly colored surface with text and figures on it, more like reading off of paper, and our visual system is more designed for this sort of viewing. I generally use this as the basis for my recommendation to use a light background.

Maureen did mention two situations when a dark background makes sense.

However, if you are using a display in a dark environment, it’s better to use a dark background as it lets you keep your eyes dark adapted. That’s why controls for airplanes and GPS units for cars switch to a dark background at night.  Usually, however, the results don’t look like symbols and text floating in the darkness of space…there’s still a sense of there being a dark surface to ground the view. So the concerns above are somewhat mitigated.

I’ve seen recommendations that white on black is better for aging eyes, and for people with low vision because it reduces the amount of light scattering and distorting the image.

She completed her thoughts with the following:

Modern displays are now bright enough that they can be uncomfortably bright. But that effect will be seen in a dark room, not in daylight. Our visual system overall adapts (adjusts its sensitivity) to the ambient brightness by several orders of magnitude. That’s how we see in a dim room and in full sunlight, and that adjustment is the reason you are momentarily blind when you walk into a dark theater after being out of doors, or painfully squint on the way out. If your display is bright enough to be significantly brighter than anything else around it, then you’ll find it uncomfortable. But we cover this by saying you should use a dark background when in the dark. And turning down the brightness remains an option for leaving the background white. Many mobile devices automatically adjust the brightness as a function of the ambient light…brighter for brighter rooms, darker for darker rooms. Saves both battery and your eyes.

There might be more to it than this. Perhaps other conditions besides those that Maureen described can benefit from dark screens, but we should determine this through research rather than following the latest fad. Visual perception can be tested using the methods of hard science. Put your trust in best practices that are based on scientific evidence. Until evidence exists, trust your own eyes. There’s a lot that you can tell just be opening them.

Take care,