Redundantly encoding time-series data in sight and sound does not speed up perception
Bissantz and Company, one of two German software vendors that makes an Excel add-in for producing sparklines (BonaVista Systems is the other), has added a new feature to their product that you should avoid. In addition to looking at a simple sparkline to see how a series of values has changed through time, you can now listen to that same information encoded as sound. Here is their explanation for the new feature:
A time-series is a pattern. It climbs upward and descends, patterns are stable or swinging, they change slowly or abruptly. The pattern is more important than the actual values…Time-series patterns are similar to sound patterns. A sound pattern climbs or descends just like values do. You can hear what you see and see what you hear. If you simultaneously hear and see the same your understanding is faster. [Emphasis mine]
Everything was fine until the last sentence. Sound patterns are indeed like time-series patterns. In fact, sound patterns are not just like time-series patterns, they are time-series patterns. They consist of values (frequencies) that unfold serially through time. The claim, however, that the addition of an audio encoding to a visual encoding of time-series information speeds up the process of understanding is not correct. It actually slows down the process. I can see the time-series pattern in an instant, because my brain processes the image in a parallel manner, but sounds are processed serially, which takes time. Audio encoding might be useful for people who are visually impaired, but for the rest of us it is not a good replacement nor even a good addition to a graphical encoding.
One Comment on “Redundantly encoding time-series data in sight and sound does not speed up perception”
The problem with this new “feature” is that is assumes that time in a time-series display is a property of the visual perceptual system. In fact, is just a property of a given image.
This is very interesting. Bertin talks about it in the first pages of his Semiologie Graphique. While the auditory perception is a linear, temporal, system (variation of sound and time), visual perception uses space (both dimensions of the plan, width and height) and variation of the image. This is how Bertin put it:
La perception sonore ne dispose que de deux variables sensibles: la variation des sons et le temps. Par contre, la perception visuelle dispose de trois variables sensibles: la variation des taches et les deux dimensions du plan, et ceci hors du temps. Les systèmes destinés à l’Å“il soat d’abord spatiaux et atemporels. D’où leur propriété essentielle: dans un instant de perception, les systèmes linéaires ne nous communiquent qu’un seul son ou signe, tandis que les systèmes spatiaux, dont la graphique, nous communiquent dans le même instant les relations entre trois variables.