Not long ago, BusinessWeek published a story titled “Data Visualization: Stories for the Information Age” by Maria Popova (self-described as a “digital anthropologist, cultural curator and semi-secret geek aggregating the world’s eclectic interestingness”). The article featured the work of Aaron Koblin of Google’s Creative Labs (self-described as an “artist/designer/programmer”). Popova and Koblin bring fresh perspectives to data visualization, but they are newcomers to the field and have both made statements about it that demonstrate their lack of experience.

Popova describes the field as follows:

Data visualization has nothing to do with pie charts and bar graphs. And it’s only marginally related to “infographics,” information design that tends to be about objectivity and clarification. Such representations simply offer another iteration of the data—restating it visually and making it easier to digest. Data visualization, on the other hand, is an interpretation, a different way to look at and think about data that often exposes complex patterns or correlations.

“Has nothing to do with pie charts and bar graphs”? I would gladly support any effort to dismiss pie charts (with a few exceptions), but the notion that bar graphs and other traditional displays of quantitative data have nothing to do with data visualization is just plain silly. No one who understands data visualization and has done any work in the field would make such a statement, nor would they go on to say that unlike quantitative graphs, “data visualization…is an interpretation, a different way to look at and think about data that often exposes complex pattern or correlations.” In truth, all visual representations of data are interpretations. The very act of selecting information and presenting it in a particular way is an act of interpretation. All forms of data visualization—whether traditional bar graphs or some of the newer animated displays—should be embraced if they bring data alive in clear, simple, and accurate ways to help us understand the stories that live therein. Let’s not be tempted to dismiss the tried and true in our excitement over the novel—or what gives the impression of being novel.

Moving on to Koblin, much of data visualization that is novel technologically works on the same principles as bar graphs, scatterplots, and line graphs. For example, Koblin created an animated display of SMS messages throughout a single day in Amsterdam. The degree to which it provides insight into this information relies on the same basic mechanism that causes bar graphs to work: using objects of varying heights to display differences in quantitative values.

As an artist and technologist, Koblin has done some interesting work. I especially like his animated visualization of worldwide air traffic throughout the course of a day. It tells the high-level story of daily air traffic and works as an effective starting point for further exploration and analysis, which would then require more conventional visualizations. At Google, Koblin has an enviable opportunity to play with data. He has produced several visualizations that are fun and useful for artistic purposes, but fewer that actually present information in meaningful and useful ways. Of the three terms that he uses to describe himself—artist/designer/programmer—self-expression as an artist and programmer appears to come through more frequently and strongly in Koblin’s work than the solutions of a designer. I certainly can’t fault him for the wonderful examples of self-expression that he’s created, but I feel that I must take issue with something he’s said about data visualization: “It’s not about clarifying data…it’s about contextualizing it.” Actually, it’s about both. Without clarity, which is sometimes lacking in Koblin’s visualizations, context can only take us so far.

In spite of its problems, I love the way that Popova concludes her article, with one minor exception:

Ultimately, data visualization is more than complex software or the prettying up of spreadsheets. It’s not innovation for the sake of innovation. It’s about the most ancient of social rituals: storytelling. It’s about telling the story locked in the data differently, more engagingly, in a way that draws us in, makes our eyes open a little wider and our jaw drop ever so slightly. And as we process it, it can sometimes change our perspective altogether.

I balk only at her statement that “it’s about telling the story locked in the data differently.” Differently than other forms of visual display that have worked in the past—like bar graphs, perhaps? To stress that the story must be told differently is to shift our assessment of what works and what’s needed to “innovation for the sake of innovation” to a degree that Popova herself warns against. As artists, programmers, and people from other disciplines and perspectives venture into the realm of data visualization, they will help us expand our horizons. Some will also be too quick to dismiss what they don’t understand as they approach data visualization, in Popova’s words, as “a new frontier of self-expression.” Those of us who have worked in the field for awhile must be open to new ideas; those who are newcomers must respect the work of those who have worked hard to make data visualization useful and effective. I welcome this collaboration.

People sometimes adopt existing terms and give them new meanings to suit their interests, leading to confusion—sometimes usefully, when things need to be shaken up, and sometimes not. Perhaps Koblin and others like him have done this with the term “data visualization” and are using it to describe something related to but in many ways different from data visualization as I know and practice it. I suspect that another term—perhaps “information art”—would better describe what Koblin and others who combine their artistic and technological interests are introducing. Manuel Lima, of the site Visual Complexity, recently made this point in a blog post titled “Information Visualization Manifesto.” Lima proceeded from there to thoughtfully and eloquently describe the characteristics that define data visualization—or more specifically “information visualization”—as distinct from other types of information display. His words are brilliant and timely. I heartily recommend that you read them.

Take care,

I recently took a look at a new book from O’Reilly Media that’s a thoughtful introduction to the general concepts of data analysis. Unlike most books on data analysis, this is not software-specific and it does not focus on some complicated aspect of statistical or financial analysis. The book is Head First Data Analysis by Michael Milton. In its Head First series, O’Reilly is trying to provide books that introduce computer-related topics in a way that speaks to the way our brains learn. These books teach the material using real-world scenarios with lots of activities to engage us in thinking, rather than merely throwing a bunch of information at us and hoping that some will stick. Head First Data Analysis fills a useful gap at just the right time. Now that organizations are beginning to take the need for effective data analysis more seriously and people of all types are becoming responsible for the task, this book presents the basic concepts in a way that is practical and accessible to all.

Milton doesn’t go deeply into any of the concepts, but that’s by design. If you’re comfortable with statistics, this is not the book for you, but if you’re one of the many people who must make sense of quantitative information as part of your job and you’ve never been trained in data analysis, this book will set your feet on the path. Milton does a good job of identifying the basics that people need to get started and explaining them in simple ways that make them immediately useful. It’s a workbook of sorts, with exercises throughout. If you really want to learn, this is the way to do it.

Take care,

I’m occasionally asked by journalists to describe actual cases when organizations have derived real tangible benefits from data visualization. When asked, I’m usually forced to answer in terms of generalities, for the following reasons:

• The nature of my work with clients—training and design consulting services—rarely gives me a chance to see the results of my work.
• On those occasions when I am able to see work that a client produced as a result of my services, I am rarely allowed to share it publicly.
• People often contact me to say how much they appreciate my work, especially my books and articles, but they rarely share specific, tangible benefits.
• When people have shared specific accounts of benefits, I’ve remembered only the gist of those accounts, almost never the details.

Even though we have plenty of evidence from years of research to support the tremendous potential of data visualization, we are lacking in specific accounts that confirm beneficial outcomes in the real world, either empirically in the form of measured results or anecdotally.

I was reminded of this frustrating blind spot today when I received yet another interview request from a journalist and imagined myself explaining once again that, although there is a great deal of evidence that data visualization works based on perceptual studies, etc., I have little documented evidence that it works in practice.

I need real stories from you who use data visualization to analyze and present data. Has data visualization led you to important findings? Has data visualization helped your organization increase revenues or decrease costs? Has data visualization increased efficiency or productivity? Have good decisions been made because information was presented visually?

Help me out here. Tell me about your experiences. Be my eyes where I cannot go myself to observe.

Take care,

In many respects, the “information age” is anything but. An overwhelming supply of data, powered by advances in technology that ignore the needs and abilities of humans, can do more harm than good. Because of what we’re learning through brain research, which has made great strides in the last decade, we now have an opportunity to do much better. Perhaps no one does a better job of explaining in broadly accessible terms what we now know about the human brain and how it works than developmental molecular biologist John Medina. What’s special about Medina’s work is that he isn’t just delivering the facts; he’s applying them in practical ways to improve our lives.

In Brain Rules: 12 Principles for Surviving and Thriving at Work, Home, and School, Medina takes us on a fascinating journey through the brain, expressing what research has revealed in the form of simple rules that we can follow to live smarter and better, and help others do the same. In the book’s introduction, Medina writes:

Most of us have no idea how our brain works.

This has strange consequences. We try to talk on our cell phones and drive at the same time, even though it is literally impossible for our brains to multitask when it comes to paying attention. We have created high-stress office environments, even though a stressed brain is significantly less productive. Our schools are designed so that most real learning has to occur at home. This would be funny if it weren’t so harmful. Blame it on the fact that brain scientists rarely have a conversation with teachers and business professionals, education majors and accountants, superintendents and CEOs. Unless you have the Journal of Neuroscience sitting on your coffee table, you’re out of the loop.

This book is meant to get you into the loop.

Our classrooms, workplaces, and homes are in many ways designed to thwart brain health, effective learning, personal fulfillment, and overall progress as a species. In many ways, the information technologies that dominate our lives today have contributed to this sad state of affairs, not because of anything inherently wrong with technology, but because most of it was developed without understanding how our brains work. As such, reliance on technology can actually make us unhappy and dumb. This is true of most business intelligence (BI) technology, which is my professional domain. All business intelligence professionals, especially those who develop BI tools, should read this book. You’ll enjoy the process, learn how to live a happier, smarter, and more productive life, and develop an understanding of the brain that will help you more effectively support the goals of business intelligence.

Take care,

P.S. The website www.brainrules.net provides a great deal of information, including videos, which you can use to preview the book. Also, Garr Reynolds, the author of Presentation Zen, put together a wonderful slideshow that you can view to get the gist of the book, especially as it applies to presenters.

In March of 2006 I glimpsed the new charting capabilities of Excel 2007 for the first time and wrote about them in an article titled “Excel’s New Charting Engine: Preview of an Opportunity Missed.” After waiting for years to see how the world’s most popular data analysis software would improve its sadly lacking charting capabilities, I mourned the opportunity for improvement that was almost entirely missed. Essentially, an entirely new charting engine in Excel 2007 replaced the old one, but what it brought with it was a fresh array of flashy visual effects that encouraged us to hide our data behind a thick layer of cheap makeup. Within two days of my article’s publication, I received an email from Scott Ruble, the person in charge of charting functionality in Microsoft Office products. Scott invited me to help the team improve the charting capabilities of the product’s next major release—Excel 2010—which will become available sometime during the first half of next year. We’ve had several conversations since, including a teleconference with the team. Early glimpses into the charting capabilities of Excel 2010 are now beginning to surface, and it appears that the opportunity to improve the product’s data visualization capabilities has once again been missed. Although I haven’t seen an advance version of the product myself, those who have tell me that it includes only one change to charting: the addition of sparklines. What a shame.

Don’t misunderstand me. I’m thrilled that a version of Tufte’s sparklines will be added. Assuming that the implementation is well designed, this will eliminate the need for an add-in product if you want to display a set of time-series values as a simple sparkline, but this is a single grain of sand compared to an entire seashore of need. No single product in the world is used more than Excel for analytics, not because it’s a good tool for data exploration, analysis, and presentation—it isn’t—but because almost everyone in the world who works with quantitative data has it. Just imagine how much the world would benefit if Excel were more powerful and better designed. I was frustrated and upset when Excel 2007 missed the mark, but now with Excel 2010 trying to assuage our misery with nothing but sparklines, I’m inclined to give up on the product entirely as a tools for data analysis. Fortunately, where Excel has failed, alternative products have emerged that deliver effective and visionary analytical abundance.

Will Microsoft play a role in the future of data analytics? Although the company boasts a business intelligence (BI) solution and even declared its commitment with the first annual Microsoft Business Intelligence Conference in May of 2007 (the 2009 conference was cancelled), only its database does anything particularly useful for BI so far. The other pieces that have been awkwardly rubber-banded together into a so-called BI solution suggest the lack of a strategy or a confused one at best, and previews of coming additions, such as Project Gemini, suggest nothing but already dated functionality for the future. I don’t have a bias against Microsoft because it’s huge and powerful; I have a progressively growing disappointment with it because of unfulfilled potential. If Microsoft seriously applied itself to the task, it could probably do some wonderful for the world of analytics. At this point, Microsoft will have to do something big, totally unexpected, and uncharacteristically well designed if it hopes to play a role in the future of analytics. I would welcome this with arms wide open, but I’m not holding my breath.

Take care,