In a recent blog post titled “Big data NSA spying is not even an effective strategy,” Francis Gouillart raised concerns about Big Data that are very much in line with mine. Gouillart’s is a refreshing and rare voice of sanity. He’s been around long enough to recognize marketing hype when he sees it, and as an independent thinker with ethics, not a shill for technology vendors, he is one among few who are speaking the truth. Here’s a sample:
The evidence for big data is scant at best. To date, large fields of data have generated meaningful insights at times, but not on the scale many have promised…Yet, for years now, corporations and public organizations have been busy buying huge servers and business intelligence software, pushed by technology providers and consultants armed with sales pitches with colorful anecdotes such as the Moneyball story in which general manager Billy Beane triumphed by using player statistics to predict the winning strategies for the Oakland A’s baseball team. If it worked for Billy Beane, it will work for your global multinational, too, right? Well, no.
The worship of big data is not new. Twenty-five years ago, technology salespeople peddled data using an old story about a retailer that spotted a correlation between diaper purchases and beer drinking, allowing a juicy cross-promotion of the two products for young fathers. Today, most data warehouses are glorified repositories of transaction data, with very little intelligence.
Working with multinationals as a management consultant, I have chased big data insights all my life and have never found them. What I have learned, however, is that local data has a lot of value. Put another way, big data is pretty useless, but small data is a rich source of insights. The probability of discovering new relationships at a local, highly contextual level and expanding it to universal insights is much higher than of uncovering a new law from the massive crunching of large amounts of data.
Read Gouilart’s article in full and pass it on. It’s time to usher in a quiet voice of sanity in this noisy, naive world of “more is better.”