Beware the Straw Man
A “straw man” is a flawed form of argument that occurs when one side attacks a position that isn’t actually held by the other side (the “straw man”) and then acts as though the other side’s position has been refuted. People usually construct straw men when they cannot legitimately refute an opponent’s position. As such, a straw man is a dishonest and fallacious form of argument, but one that can be persuasive when the audience is not aware of the facts.
I learned about straw men as an undergraduate majoring in communication studies. I loved the course that I took in argumentation and debate back then because I found the rules of logic elegant, interesting, and easy to understand. I vividly remember, however, that most of my classmates didn’t take so naturally to these principles and frequently struggled to make their case. I’m ashamed to admit that I took far too much pleasure in tying my opponents into logical knots and luring them into logical traps.
Since those bygone days of youth, I have expanded what I learned in college by keeping up with work in the fields of critical thinking and brain science. I am now familiar not only with the rules of rational argument but also with many causes of flawed thinking. I have found, to my great disappointment, that this is not common knowledge, even among scientists and analysts. I am no longer surprised when academics in the field of information visualization—doctoral students and professors—conduct studies that are flawed in obvious ways.
I was prompted to think about straw men recently when I encountered a couple on the Web that were apparently constructed to fault the work of people like me who teach data visualization best practices. The first appeared in a recent series of articles about data visualization on the Harvard Business Review’s (HBR) website. I was invited to contribute an article to this series, but unfortunately didn’t have the time. I wish I could have participated, however, to correct the portrayal of business-related data visualization as skewed toward elaborate infographics rather than the simple uses of quantitative graphics that make up around 99% of the data visualizations created in organizations. The straw man that I noticed was constructed by Amanda Cox of the New York Times. I greatly admire the data graphics of the New York Times, including Amanda’s work in particular. Cox is an articulate spokesperson for journalistic uses of data visualization. For this reason, I was surprised when I read the following interaction in HBR’s interview with Amanda (emphasis mine):
[HBR]: It seems like there’s more focus on trying to get data viz to go viral than to make it “matter.”
[Amanda Cox]: There’s a lot where not much actionable comes out of it. I don’t know if the ratio is different from the ratio of bad writing to good, or bad restaurant openings to good, but I think it’s an important idea to focus on. There’s a strand of the data viz world that argues that everything could be a bar chart. That’s possibly true but also possibly a world without joy.
I appreciated almost everything that Amanda said except the two sentences that I’ve highlighted above, which appear to be a jab at data visualization practitioners who promote the use of simple graphs over some of the elaborate (but often ineffective) infographics that routinely appear on the Web. Amanda’s statement is a straw man. No one “argues that everything could be a bar chart.” Anyone who did would not only be robbing the world of joy but also of meaning. Bar graphs are one effective means of displaying data among several, and they are only appropriate for particular data sets and purposes. I’m not sure why Amanda felt compelled to insert this little goad of a comment in the interview. If she has an actual case to make, she can surely do better than this.
On April 17th, I encountered a similar straw man constructed by Nathan Yau in his blog (emphasis mine):
Data is an abstraction of something that happened in the real world. How people move. How they spend money. How a computer works. The tendency is to approach data and by default, visualization, as rigid facts stripped of joy, humor, conflict, and sadness—because that makes analysis easier. Visualization is easier when you can strip the data down to unwavering fact and then reduce the process to a set of unwavering rules.
The world is complex though. There are exceptions, limitations, and interactions that aren’t expressed explicitly through data. So we make inferences with uncertainty attached. We make an educated guess and then compare to the actual thing or stuff that was measured to see if the data and our findings make sense.
Data isn’t rigid so neither is visualization.
Are there rules? There are, just like there are in statistics. And you should learn them.
However, in statistics, you eventually learn that there’s more to analysis than hypothesis tests and normal distributions, and in visualization you eventually learn that there’s more to the process than efficient graphical perception and avoidance of all things round. Design matters, no doubt, but your understanding of the data matters much more.
I agree with everything that Nathan says here, but not with what he implies in the text that I’ve highlighted. His comment about “efficient graphical perception and avoidance of all things round” appears to be a direct reaction to my position, but one that he’s morphed into a straw man. No one argues that there isn’t more to data visualization than perceptual efficiency and circle avoidance. (I suspect that Yau’s phrase “all things round” refers to an article that I wrote in 2010, “Our Irresistible Fascination with All Things Circular.”) No one who promotes the importance of efficient and accurate graphical perception argues that design matters more than understanding. In fact, it is our concern that people understand data clearly, accurately, and as fully as possible that leads us to teach people how to present data graphically in ways that work for human perception and cognition. There is indeed much more to data visualization than a rigid set of design rules, which is why, when I teach design principles, I do so in a way that enables my students to understand how and why these principles work so they can apply, bend, and sometimes break the rules intelligently.
What’s ironic about Yau’s claim is that he often features infographics as exemplary that are beautiful or otherwise eye-catching, but yield little understanding. Such examples can easily be found in his lists of the best data visualizations of the year. Given his training as a statistician, I’ve always found this puzzling.
Making data visualizations perceptible is not all there is, but it is certainly an essential requirement if we want people to understand what we’re trying to say. I’m sure that Cox and Yau agree, but they seem willing at times to sacrifice perceptual effectiveness for visual allure. When they do, understanding is diminished. There is no reason why perceptual effectiveness and visual allure cannot coexist. Leaders in the field of data visualization don’t always agree, but when we disagree and wish to state our case, we should build it on solid evidence and sound reason. Dismissive remarks and thinly veiled insinuations that aren’t accurate or backed by evidence don’t qualify as useful discourse.
Take care,
8 Comments on “Beware the Straw Man”
I’m more worried about colleagues constantly using the term ‘straw man’ to mean ‘demo system’ – “We’ll just build them a straw man report…”
Boils my blood.
Good article!
You could actually use another rhetoric/reasoning tool (an absurd analogy) to dismiss that comment, which is what I tried to do briefly here: http://www.thefunctionalart.com/2013/04/a-bubble-chart-or-hairball-would-be.html
Are there any texts you’d recommend on debate and argument.
“Design matters, no doubt, but your understanding of the data matters much more.”
Doesn’t one directly affect your ability to do the other?
Andrew,
Absolutely. Good design facilitates understanding. Bad design undermines understanding. Extraneous design (i.e., design that directs you to focus on or to do things that are irrelevant to the task at hand) is bad design. It is in the latter sense that many infographics fail. They direct people to focus on visual content that is not relevant to the information. It is not useful to engage people unless you do so in a way that leads them to examine and understand the information that you are trying to communicate.
Yau’s statement, “The tendency is to approach data and by default, visualization, as rigid facts stripped of joy, humor, conflict, and sadness—because that makes analysis easier.”, is another part of the straw man that frosts my cookies. The idea that “data people” have no heart or creativity while “visual people” are lovable free spirits is widely accepted, but that does not make it correct.
Data are facts and are true regardless of the emotions that go with them. Assigning meaning to the facts is what makes them human, and that is part of the analysis. If your data is not factual then your analysis and visualization are worthless. Gleaning valuable meaning from data is the creative work of the Data Scientist. Visualization, when done well, communicates the meaning. The “data people” are the ones who bridge the gap from facts to knowledge and understanding, which is arguably the more creative of the two tasks.
There’s quite an interesting book (to be read with a pinch of salt) called Academically Adrift. One of the assertions in this book is that students are graduating from university without critical thinking skills. Indeed ‘teaching to the test’ and ‘assessment as teaching’ have lead us to a position where marks are higher than ever, and more people than ever are achieving at a higher academic level, but still we have people who cannot think critically. They confuse correlation with causation, average averages, think that A must have caused B because it happened first, see patterns in randomness and seek to maximise KPIs without the first consideration of the consequences. Yet many of these graduates are ridiculously successful in their careers.
In the book ‘The Tiger that Isn’t’ (UK focused, but still generally interesting) suggests that all analysis should begin with two questions: ‘what are we counting?’, and ‘is it a big number?’. Folk just don’t do this — instead they leap heroically to utterly flawed conclusions as a result of gut instinct — instinct that they have been rewarded for at university and their subsequent career. We’ve gone past the age of reason to an age of proof by status and beauty! Graphicacy appears to be a second or third order problem after the issues of applying reason and being numerate…
An excellent article that inspired me to write about it on Freakalytics. I suppose it is all about what gives us joy. Personally, I take great joy in helping people understanding the data, so I don’t understand why bar charts are joyless.
However, as an analyst, statistician and data miner at many companies, I can tell you what isn’t joyful- standing in front of a group of executives and conveying incorrect answers because they misread your “joyful chartâ€. How about showing a chart in a meeting and saying “as you can plainly see†when no one can see it. Having this problem across several meetings will eventually marginalize your work as an analyst since you are making points they can’t see! Most business analysts work hard to earn credibility and trust through quality analysis, clear explanations and well-reasoned recommendations. Why stack the deck against them with misguided visual analytics advice?