Failures in Analysis Are Failures in Thinking

Psychology of Intelligence Analysis, Richards J. Heuer, Jr.,
Center for the Study of Intelligence, CIA, 1999.

In any field of study, among the many written works that inform it there are a few that stand out as pillars of wisdom. In the field of data analysis, one of those pillars is the book Psychology of Intelligence Analysis, by Richards J. Heurer, Jr., who spent 45 years supporting the work of the CIA. Even though this book focuses on intelligence analysis, the principles and practices that it teaches apply to data analysis of all types, including the analysis of quantitative business data. Heuer believes, as I do, that the primary failures of analysis are less due to insufficient data than to flawed thinking. To succeed analytically, we must invest a great deal more of our resources in training people to think analytically and equipping them with tools that effectively support the process.

According to Douglas MacEachin, former CIA Deputy Director of Intelligence:

Dick Heuer makes clear that the pitfalls the human mental process sets for analysts cannot be eliminated; they are part of us. What can be done is to train people how to look for and recognize these mental obstacles, and how to develop procedures designed to offset them.

Throughout the book, Heuer identifies these pitfalls in thinking and suggests guidelines and procedures for overcoming them, and he does so in clear prose that anyone interested in the topic will find accessible.

People construct their own version of “reality” on the basis of information provided by the senses, but this sensory input is mediated by complex mental processes that determine which information is attended to, how it is organized, and the meaning attributed to it. What people perceive, how readily they perceive it, and how they process this information after receiving it are all strongly influenced by past experience, education, cultural values, role requirements, and organizational norms, as well as by the specifics of the information received.

Unlike some who would shroud the analytical process in a mystique of lofty terms and obscure references, Heuer extends a friendly, helping hand. He does so because he believes that these skills can be learned.

Thinking analytically is a skill like carpentry or driving a car. It can be taught, it can be learned, and it can improve with practice.

To give you a better idea of the book’s contents, here’s the outline:

Part 1—Our Mental Machinery

Chapter 1: Thinking About Thinking
Chapter 2: Perception: Why Can’t We See What Is There To Be Seen?
Chapter 3: Memory: How Do We Remember What We Know?

Part 2—Tools for Thinking

Chapter 4: Strategies for Analytical Judgment: Transcending the Limits of Incomplete Information
Chapter 5: Do You Really Need More Information?
Chapter 6: Keeping an Open Mind
Chapter 7: Structuring Analytical Problems
Chapter 8: Analysis of Competing Hypotheses

Part 3—Cognitive Biases

Chapter 9: What Are Cognitive Biases?
Chapter 10: Biases in Evaluation of Evidence
Chapter 11: Biases in Perception of Cause and Effect
Chapter 12: Biases in Estimating Probabilities
Chapter 13: Hindsight Biases in Evaluation of Intelligence Reporting

Part 4—Conclusions

Chapter 14: Improving Intelligence Analysis

If your job involves making sense of data to support decision making, you owe it to yourself and your employer to read this book. It won’t take long to read and it needn’t cost you anything, because the book can be downloaded as a PDF for free from the CIA’s website. If you like it as much as I do, you can also purchase a bound version.

Take care,

One Comment on “Failures in Analysis Are Failures in Thinking”

By Mike Moroze. March 19th, 2009 at 11:32 am

Thanks for pointing to this book! Looks like a great reference that can be used in many contexts. I’ll know more after I’ve read it ;-).