Looking for relationships in quantitative data is fun. Contrary to the reputation of quantitative analysis as deadly boring to all but the pocket protector gang, most people find it engaging when the process is made simple, the quantitative relationships are brought to life visually, and the information is interesting. When Martin Wattenberg, an exceptional talent in the field of information visualization, developed a little Web application called NameVoyager to promote his wife Laura’s book The Baby Name Wizard, he had no idea how popular it would become among a broad range of people, not just those trying to choose a name for their babies. All sorts of people found their way to NameVoyager and had loads of fun with it. Initially, most people looked up their own names to see how they had changed in popularity from the 1880s until today, but most moved on to explore and make discoveries about the changing popularity of name after name, examining the historical ups and downs in their popularity, and even testing sets of names to see if their popularity was correlated in some way. People would stand around computers in groups calling out “look up Sally now” and the like. People were sending emails to one another excitedly telling them to examine such and such to see the interesting historical trends. Although most people might not have called what they were doing “quantitative analysis,” in this case exploring and making sense of time-series sets of values that measured the popularity of names, this is precisely what they were doing, and having fun in the process.

The guys at www.swivel.com—data nerds by their own admission—recognized this fact and have created a playground for the public to have fun examining historical quantitative relationships. I’m inclined to appreciate and support any effort that invites people to think quantitatively and analytically. I like the intentions of the folks at Swivel, but have concerns about their execution.

Most of my current work in data visualization is a response to the sad fact that it is the rare businessperson who knows how to communicate quantitative data graphically. They need to be taught the necessary skills, and so I write books and teach workshops to address this need. My concern with Swivel is that this site makes it easy for people to explore and make discoveries about data, but they don’t teach them the skills that are required to do it meaningfully, nor have they built best practices into their graphical playground to prevent people from reaching wrong and potentially absurd conclusions. How long will it be before USA Today begins to feature such amazing discoveries as the negative correlation between wine consumption and violent crime and promote it as a causal relationship, using the graph from Swivel to validate its claim.

Swivel was recently released in preview mode and I’ve had but little time to explore it. It didn’t take long, however, to discover several problems. The first time that I read about Swivel and followed a link to the site, the following graph was being featured:

My eyes were immediately drawn to the intersection of lines around the year 1995 and to the peak in the one line around the year 2002. As soon as I tried to interpret the data, however, my efforts were frustrated because my poor eyes could not match up the colors in the legend to the colors of the lines, even though I was wearing my brand new reading glasses and had expanded the graph to maximum size. Many of the lines appeared to have the same or nearly the same colors.

I started searching for a way to identify the individual lines. I tried pointing to and clicking individual lines to no avail. I soon discovered, however, that I could click a label in the legend to make something happen, but my hope that the corresponding line would be highlighted in some way was dashed when the graph was replaced with one that included only the line that I selected in the legend. I had no way to identify individual lines in the context of the whole.

I gave up on understanding which line was which and decided to see if I could discern anything else meaningful from the graph. I noticed that the percentage scale on the Y-axis included negative percentages, which caused me to wonder exactly what the scale represented. According to the Y-axis title, I was looking at “relative values.” Relative to what? I had to go exploring again for an explanation. Eventually I discovered that I could choose to edit the graph, which took me to a different view where I could select from three quantitative scales: absolute values, relative to average values, and relative to range values. “Relative to average” was selected, so I surmised that the average value was calculated as the mean of all values across the entire range of time (1970-2004).

I could now make some sense of what I was seeing, but I couldn’t quite shake the concern that my understanding of how the average was calculated might be wrong or that it might have been calculated incorrectly. For one thing, I didn’t know if the average was calculated for each line independently, based only on its values, or on all the data in the graph. My suspicion about the accuracy of the calculation was aroused by the fact that this graph was composed of unequal intervals of time. Upon examining the data table, I found values for the following years: 1970, 1980, 1985, 1990, 1991, 1992,…2004. Intervals varied from 10 years to one year, but nothing in the graph informed me of this fact, which must be taken into account when examining the data and in calculated averages across this period of time.

I went exploring again, shifted back into edit mode, and this time changed the quantitative scale to “relative to range.” When I viewed the results of this change in a new version of the graph, once more the Y-axis title proclaimed “Relative Values.” Given the fact that there are two types of relative scales available, this wasn’t very informative and was in fact potentially misleading. I noticed that in this new version of the graph, several but not all “ways of dying” began with their lines at 100% in 1970 (see below).

I guessed that “relative to range” must mean relative to the highest and lowest value across the entire range of time, either for each line independently or for the full set of lines, but if this were true, wouldn’t at least one line dip to a value of 0% at some point? It would be nice if I didn’t have to guess at the meaning of the scales. One fundamental rule that I teach regarding quantitative communication is that you should include an explanation of everything that isn’t obvious.

I began to wonder about the one remaining type of quantitative scale: absolute values. I was curious about what would happen if the sets of values that were being compared in the graph didn’t share the same quantitative scale, for example, per capita income in dollars and infant mortality in number of deaths. How would unlike scales be handled in a single graph? So I went searching for a graph that might illustrate this situation, and found the following simple example that compares two variables: per capita sales of wine (in dollars, I assume) and incidences of violent crime (see below).

It took me a while to realize that the line for “Per Capita Beverage Wine” wasn’t omitted, as I first assumed, but was rendered almost invisible because it was stuck at a value of zero across the entire graph. Because the quantitative scale is expressed in thousands, per capita wine sales don’t even register. Despite the appearance of this graph, wine sales actually did vary significantly between 1970 and 2004, but there is no way to show this when the quantitative scale is set to accommodate the much greater values associated with violent crime. The rule that was broken this time is that you don’t combine data with unlike quantitative scales into a single graph with a single scale. Instead, to compare them you create multiple graphs, one for each quantitative scale, and place them one on top of the other so that the same points in time align and patterns of change through time can be easily compared.

I’ll end my critique of the site here, which has perhaps already gone on too long for a blog, but I want to finish with an expression of hope for Swivel. I think these guys are trying to do something worthwhile. I applaud their effort. I can’t imagine the work that has gone into the development of this site. Rather than discourage their efforts, I want to encourage them to make the site more effective. The problems that it exhibits must be solved in two ways:

1. By providing instruction to those who use the site in how to explore and interpret data meaningfully.
2. By incorporating best practices into the site, drawing on the knowledge of experts in the field of information visualization.

I noticed that one of the three advisors to the Swivel team is a professor in the Computer Science Department at the University of California, Berkeley. I know of at least three experts in information visualization at this university: Maneesh Agrawala (also in the Computer Science Department), Marti Hearst (School of Information), and myself (Haas School of Business). The Swivel team should reach out to experts in this field, either at Berkeley where they’ve already established a relationship, or elsewhere. I believe this venture is too great an opportunity to squander through lack of critical expertise. This information visualization site will succeed only if it is rooted in the best practices of information visualization. Without this expertise, it will quickly degenerate into the ridiculous.

Take care,