The June/July 2008 issue of Scientific American Mind includes an article entitled “Your Inner Spam Filter” by Andrew W. McCollough and Edward K. Vogel of the University of Oregon. In it they explain that superior abstract reasoning appears to be related to better use of working memory and that the difference between those who reason effectively and those who don’t might be due to differences in the brain’s ability to filter out information that is not relevant to the task. Research has demonstrated for years that working memory is limited to about four chunks of information at a time. You might be surprised by how little we can hold in working memory, the area in our brains where information is temporarily stored while we’re thinking about something. Better use of working memory’s limited capacity results in better abstract reasoning—the kind of reasoning that handles data analysis. So, the analytical process is improved by the brain’s ability to filter out irrelevant information, a function that works a bit like working memory’s spam filter.

Data analysis software, in an effort to support the process, should eliminate all non-essential content, thereby reducing the need for an analyst’s brain to filter it out. By doing so, good software will help to level the playing field between analysts whose internal spam filters vary in quality. The current trend in data presentation and analysis software to display information using flashy visual effects, such as 3-D charts with lighting effects to make them look photo-realistic, rather than displaying the data alone in simple, clear, and meaningful ways, is working against the needs of analysts. As Edward Tufte wisely wrote in back 1983, “Above all else show the data.”

Take care,