Big Data, Big Overload
Yesterday morning I had a Big Data experience when I visited the Louvre Museum in Paris. Within 15 minutes of arriving I was ready to run screaming from the glass pyramid. Why? Because I was overwhelmed. Room after room of artistic works totaling in the hundreds of thousands, each magnificent, was too much. I found it nearly impossible to appreciate a single piece when I was surrounded by so many others. Pick any one of those glorious works and place it before me in a quiet room with good lighting and I would study it for hours. Give me something to read that describes the work—the artist, the medium from which it was created, the historical context—and I would appreciate it for an entire day. Place it among thousands of its brethren and I might fail to see it altogether.
We are surrounded by data. In our present day of so-called Big Data, there is more and more of it every day. Anyone who has ever actually worked with data in an effort to make better decisions knows that most of the data that surrounds us is noise. It’s useless. We seek the signals that reside here and there in the midst of the noise. While I stood there in the Louvre this morning, every piece of art was a masterpiece in its own right—every piece a signal—but to me they were all noise because there was too much for my senses to take in or my brain to fathom. Yes, even signals become noise when we’re overwhelmed. I tried desperately to fix my attention on a single piece, but over and over again I failed. I couldn’t shut out the other voices constantly invading my senses yelling “Look at me!” Yes, I saw the Mona Lisa with her enigmatic smile from behind the barrier while being jostled by the photo-taking crowd, but I couldn’t connect with her or the genius of da Vinci, whose work I so admire.
Others in the Louvre yesterday added the Mona Lisa and a host of other works to their lists of important encounters and no doubt felt enlarged by their experience. But were they enriched in a meaningful way? Did the artists’ voices reach their ears through the din? Were they awakened or did they merely roll over in their sleep?
Data becomes information only when it informs. For that to happen, we must find a signal in the midst of the noise and study it closely enough to understand it. This takes time. This takes attention. This takes skill. Only when this occurs has something useful entered our minds.
Take care,
6 Comments on “Big Data, Big Overload”
That’s why I prefer to visit smaller museums. Less competition for my attention. Also, there is less of an assumption of what I should be looking at. (How many people miss important pieces in the Louvre because all the signs point to the Mona Lisa?)
So, to continue your analogy, it looks like the “brave new world” of Big Data still needs datamarts. To find the signal in the noise, start by filtering out the noise that doesn’t match the signal. (Or, in art terms, “remove everything that doesn’t look like an elephant”.)
Of course, there’s also the tricky bit you mention at the end of the first paragraph: the “something to read” that gives depth and context to your signal. This seems to be far easier to find with art than with data.
So true … I now realize that I had the same experience a few years ago in the Louvres, but I didn’t make the link with overwhelming information and the buzz around “Big Data” that we have today. Less is more, indeed !
It’s great to have a voice like yours in this world that becomes so much more difficult to make sense of it. You showed me for the first time the Data -> Information -> Knowledge -> Wisdom model: we still seem to struggle with the step from I to K, and “Big Data” will probably not help us that much …
Lost in the clutter:
“Place it among thousands of its brethren and I might fail to see it altogether.”…so true of the signboards that litter our streets…literally so many of them that you don’t notice one.
The latest “gismo” that everyone has here now on all the brochures, signs etc is “Like us on Facebook” like its some sort of licence to play.
Like your whip holder …pretty obsolete when everyone has it or when facebook is replaced.
Reminds me of the spreadsheets with hundreds of rows, all different colors. Instead of being useful, my eyes just glaze over. I’m sure that is why management keeps asking for more information, even though they have everything they need. They can’t find the signal with all that ‘noise’.
You’re right, big data is mostly noise (or a noisy marketing campaign) and can distract when you want to look at something specific. However lets not write it off completely.
If you are a museum director would you want to know any of the following:
Was your sense of being overwhelmed greater or less than the last time you visited?
Where in the museum did you feel least overwhelmed?
How much time did you have to see the Mona Lisa? How many people were there on average?
Or if you prefer to talk about spreadsheets with lots of different colours, what is the predominant one, red would traditionally be bad, green would be good (ignoring the 15% of the population who have trouble telling the difference).
Listen to the volume and frequency of the noise and see if there are broad patterns, there can be signals in there. That’s not to say you just take whatever signal you see at face value but it can help you to decide where to look in more detail.
Big Data Big Picture.
Damien,
You’re concerned that I should not write off Big Data completely, but there is no “it” to write off. Big Data is nothing but a marketing campaign. Everything that you advocate is done with data, not something that is supposedly new and unprecedented called “Big Data.” Big Data is just another name for data as usual. When was data not big?