I’m so sorry that you had to take the time to address Mark’s correspondence(s).

Unfortunately, some people’s passions blind them (yes, pun intended) to the realities and associated implications of a course of action they feel will rectify inequalities, be they physical disabilities, economic, or other.

I personally don’t have the scientific background to know how the eye and brain process everything but as a Usability Architect I have some basic understanding of this from reading research done by folks in the usability realm…and everything you said makes total sense.

And, I’m glad you made clear that the accessibility features that can help the blind, while worthy, do not provide dashboards. Hopefully Mark will get that distinction since it was lost on him from the start.

Mark,

If you’d like, I’ll gladly share everything that you wrote to me privately. As you know, it was only after I pointed out your false statements and rude accusations that you backed off and said that you were merely looking for advice.

In your document “A Guide to Accessible Content for Oracle Business Intelligence Suite 11g” (2012), you recommend that designs be designed such that they can be read by screen readers and thus made accessible to the blind. At no time did you mention that in doing this, the dashboard for those with sight will be reduced to something ineffective and unworthy of the name. As I wrote in my blog article, you should have encouraged readers to create a separate display for access by screen readers rather than altering the design of the dashboard itself. Solving the problem in this manner would give the information contained in the dashboard to everyone in the best manner possible. Even though a text-based version of the information cannot come close to the information contained in a dashboard (i.e., a well-designed visual display), it could be structured specifically for ease of use by screen readers.

Regarding the screen shot that I showed, it supposedly labeled the sections of a dashboard, but what it displayed was in fact not a dashboard. That’s a bit confusing, isn’t i? The use of crosshatching in graphs to solve the problem of colorblindness is never a good solution. You should not have recommended it.

As a BI software vendor, Oracle is responsible for providing products that can be used to explore, analyze, and communicate information as effectively as possible. As the writer of a guide for using Oracle’s BI products, you share in this responsibility. What you have provided, however, will undermine the usefulness of dashboards. I’m confident that this was not your intention. You now have an opportunity to fix the problem by revising the guide.

Mark,

I appreciate your humility and willingness to find better ways to support accessibility. I’ll be in touch. In the meantime, let’s see what others have to say. A good starting point would be to read Colin Ware’s book “Visual Thinking for Design” to better understand visual perception and what’s lost when we design a visual monitoring display such as a dashboard for accessibility to screen readers.

As I was reading some articles about dealing with color-blindness of some users I was wondering about this issue. Thanks for answering some of my questions and some that I had not even thought of yet.

~Tricia

Mike D,

While it is true that screen readers are limited in what they recognize (text only), adding the ability to read values in a graph and perhaps even describe the patterns won’t solve the problem. People with vision can look at a graph and often detect what’s significant and therefore worth examining or comparing in an instant, but software wouldn’t be able to do this because it cannot perceive or think visually. The time of blind users would be wasted listening to long series of values and description of patterns, most of which wouldn’t be useful. The fundamental problem resides in the fact that our senses are not interchangeable. They perceive differently and therefore the information contained in one channel cannot be fully expressed using a different channel. We will serve the needs of the blind better by finding the best ways to communicate the essential information that’s contained in a dashboard using sound or touch as well as possible rather than trying to get technology to translate visual information into verbal information. Perhaps an appropriate analogy would be an attempt by a computer programmer to read the pixels in a painting and translate them into sound, which would result in noise, when he should instead seek to find a musician to express the essence of the painting musically.

Mike D,

Perhaps it is not that I lack vision but that you’re vision is naïve. The advances in computer vision and image processing that would be needed to describe the graphical contents of dashboards in ways that would enable those without sight to use the data as intended do not exist and perhaps never will. Human visual perception and visual thinking (e.g., pattern detection and interpretation) is unique and powerful. Work in computer vision is primitive compared to human vision and might remain so. It would actually be fairly easy for a computer program to read the values in a graph, because those values are encoded geometrically (position, size, length, etc.) in ways that a computer could easily decode. Actually, if I’m not mistaken, programs of this sort already exist. The problem isn’t in decoding the values in a graph, but in understanding the meanings that are encoded in patterns and through comparisons of patterns that a computer cannot decipher, and even if it could, it could not communicate in words or sounds.

Many people who are blind have developed extraordinary abilities, especially using their senses of hearing and touch, sometimes in ways that compensate for aspects of vision (e.g., echo-location for sensing the environment) and sometimes to do things that can only be done with hearing or touch. There is a blind fellow who does close-up magic with playing cards that is unprecedented, because he has learned to do things with hearing and touch that could never be done with vision; things that allow him to fool the eyes of those with sight. As talented as he is, however, he will never be able to analyze data as I do visually, because many aspects of vision cannot be duplicated by his other senses.

Your hope that screen readers could be improved to provide better accessibility for the blind has merit, and I share your hope, but it did not stem from anything that Mark said. At no point during my interaction with Mark did he express concern about the limitations of screen readers or suggest improvements. Had he done so, I might have addressed the possibility. Improved screen readers would provide a benefit in general, but would have no bearing on the points that I’ve made about dashboard design.

Perhaps my idea is naive, but from belief in the impossible comes innovation. I don’t really think my idea is that far fetched though. I never tried to say that it would be a replacement for how people visually process a dashboard.
I was never much taking a side in your argument with Mark either. If anything I thought I was agreeing with you that dashboard design shouldn’t be changed.
Mostly I thought it was an interesting idea that was a different perspective on the issue being discussed. Perhaps I’ve just veered a bit too far off topic.

Marty,

Your observations are perhaps a bit off topic, but they’re important nonetheless. Mobile devices have driven a frenzy of business intelligence development that in many cases is absurd. The “human scale” is a reality to which technology must adapt. Unfortunately, too many technological developments are ass-backwards in that they force humans to adapt to them, which doesn’t work because it isn’t possible. The evolution of fundamental human ability occurs over eons, not years. Small mobile devices are wonderful, but they are only useful for tasks that can be done with tiny displays. Even though I show an iPhone-sized dashboard of sorts on the cover of the new edition of Information Dashboard Design, I am not suggesting that mobile devices can replace devices with larger screens. They can’t. Only a simple and limited dashboard can be viewed on an iPhone. Exploratory data analysis cannot be done on a small device, because it requires a lot more space to display multiple views of data simultaneously. As with any new technological development, in its infancy everyone goes crazy and vendors enjoy the feeding frenzy and the resulting revenues, but in time the marketplace finally recovers its sanity as people begin to recognize the limitations and proper uses of these new technologies. This will happen with mobile devices, but we’re still in the midst of the feeding frenzy.

Hello Ty,

The solution that you’ve imagined has been considered but rejected, because it won’t work. The problem resides in the difference between audio/verbal/tactile perception and visual perception. Braille must be read serially, one letter at a time. Imagine a series of 12 monthly sales values encoded as a sparkline. Visually, we can perceive that sparkline all at once, as a single pattern, in parallel. We can understand it as a whole. Reading the same 12 values encoded in Braille would not only be slow, it would also never make it possible for the reader to construct a sense of the whole from the individual values. Let’s take it up a notch. Imagine trying to compare two lines in a line graph by reading the values in Braille. It can’t be done because you cannot construct a clear sense of the whole from the parts and hold the two sets of values in memory in a way that would allow comparisons. Information encoded as sounds has the same problem. The tones that might be used to encode a series of values must be heard one tone at a time, serially.

You see, I don’t dismiss the possibility of fully translating the information in a dashboard into something that’s accessible to the blind due to a lack of vision or imagination. My objections are rooted in an understanding of human perception. As long as human perception continues to work as it does now, this translation of information from graphics to words, Braille, or sounds simply won’t work in a way that duplicates or even comes close to the visual experience.

Ute S.,

Hearing from people who are blind would indeed be useful, but it is not true that discussions without them are pointless. On the contrary, many worthwhile points have been made in this discussion. We have not already heard from anyone who is blind (at least I assume this is the case) because people without vision do not frequent blogs about data visualization, for obvious reasons.

The two questions that you’ve posed on behalf of the blind have already been answered. A description of the data in words cannot duplicate the information that’s contained in a well-designed graphic. If by “sufficient” you mean that it the verbal description would provide all of the content of the graphic, then it will never be sufficient. The point that I’ve made about the advantage of tables as an alternative display for those who are blind is that the essential information that’s provided could be better organized for screen readers if presented in tables. Structuring the data in this way for this specific purpose will produce a better result. Hearing from those who are blind might clue us into questions that we haven’t thought to ask yet, which would be useful.

Kris,

The fact that the visual cortex is used by those who are blind to process information sensed through touch when reading Braille does not mean that they perceive and process that information in the way that visual information is processed. Our brains are plastic (malleable and adaptable) in that areas ordinarily used for one thing can be repurposed to do other things when not being used in the usual way. Think of the brain as a mass of circuits that can be wired in a number of different ways depending on experience and sensory deficits (e.g., lack of vision).

Our eyes take in information at a much greater bandwidth than touch or any of the other senses. We know that reading Braille through touch is a slow, serial process. This sensory process, which takes place before the information is processed by the brain, is not sped up by the fact that the visual cortex processes it. You’re asking a reasonable question, however, which I can’t answer definitely: “Once tactile sensory data gets to the visual cortex, is it processed in parallel then?” Perhaps, but I doubt it, because the bandwidth at which Braille is sensed and the one-dimensional nature of the information, would not benefit from parallel processing. By “one-dimensional” information, I simply mean that it consists of words alone. Unlike vision, which includes multidimensional information (2-D position, size, length, width, hue, saturation, texture, etc.), there is no need for parallel processing to decode a multidimensional stream of information from Braille and then weave it into a picture. Is it possible that information encoded as Braille could be used by the visual cortex to form multiple patterns that could be simultaneously seen and compared, such as lines in a line graph? Perhaps, but I’m not aware of any evidence that this is the case.

The article that you cited is interesting in that it uses terms for sight–picture and image–several times to describe non-visual perception. Expressed in this way, the article suggests that those who are blind construct a visual representation of the information that they sense through touch or sound. Unfortunately, however, we don’t know how this information is represented in a blind person’s brain and how that representation compares to that of someone with sight. A statement in the article by Paul Gabias, a psychology professor at the University of British Colombia who is blind, seems to draw an unwarranted conclusion:

“If you know that blind people know where to put their plates on their table, and you know that blind people deal with tables in the exactly the same way you do, then you presume that they imagine them in the same way you do. You have got to presume that what’s inside their head is like yours.”

In fact, we cannot and should not presume that what’s inside the heads of blind people is the same as what’s in the heads of people with sight. Knowing where objects are located on a table as well as the shapes and textures of those objects, which enables people who are blind to use those objects in a way that is not noticeably different from people with sight, actually tells us little about the way that information is represented in their heads. A conceptual construct exists, but we don’t know how it compares to one that is built from vision.

To clarify, I wasn’t suggesting a hypothetical device that would translate graphics to braille. Rather, I was recommending a device that simply embosses the graphics entirely, as-is. The burden at that point would be on the blind to learn how to “read graphics” or simply to “Experience The New”. If I were blind, I’d probably figure it out. (who knows… that’s a bold statement.. :)

Carry on. Interesting conversation to be sure.

Neil,

We don’t need to wait for completely different tools” to be developed to determine the benefits of visual perception compared to our other senses. Although we’re learning more about perception every day, we already know a great deal. I’m not arguing that visual perception is a superior channel for all information. Each of our senses has its unique strengths. I am arguing, however, that for pattern perception and interpretation, as well as pattern comparison of the types of information that we display in dashboards, vision trumps all. This is not an opinion; this is well established by science.

Your radio play analogy is not appropriate. Yes, someone can listen to a play on the radio and have an entertaining experience that is worthwhile, even though it lacks the visual elements of the same play presented on stage. Words can be used to paint pictures in the imagination of listeners. The goal of a radio play, however, is not to convey a specific collection of information for particular purposes, but this is the goal of a dashboard. When we display information on a dashboard, it isn’t enough for someone to get something—anything—worthwhile from it. We want people to get a clear and accurate understanding of particular information.

A VR suit that communicates information to our senses of touch, hearing, smell, and taste, but not sight, will remain limited by the abilities of the human brain. The primary limitation that would defeat us in this venture is the fact that only vision processes information in parallel rather than sequentially, making it possible to simultaneously perceive patterns made up of a great deal of data that could never be stitched together in our brains from a sequential stream of facts. It is wonderful to dream about the great things that we might achieve through technology, but dreams that don’t take into account what is and isn’t possible will never be anything but fantasy. If and when we discover perceptual abilities that are presently unknown, then we can build on them. That’s where our efforts should be focused, not on constructing systems from parts that we already know don’t work.

I have spent many years (and been recognized for) working to identify ways that visually-focused electronic end products (such as dashboards, graphs and presentations) can be made more accessible. Not only for persons using assistive technology but all persons. None of us are exempt from aging and we’ll all face physical challenges at some point.
I do agree with Stephen mostly. Providing an accessible alternative to the dashboard is currently the best option for persons using a screen reader. Not because I think it’s impossible to make a dashboard accessible, because technically it is, I just don’t think the technical answer is the best answer.
It’s not a case of whether you can, it’s whether you should.
Providing an easily navigated alternative with a clear summary (of the most important information needed to achieve one or more objectives) and the data behind the graphs laid out in an accessible table format is one example of a more effective alternative. It allows the blind user to absorb the point of the dashboard as well as the opportunity to review the details if they so desire. Even if the way they absorb the information is different, I think it’s still important to allow them the opportunity as much as possible.
I’ve learned that blind users can absorb much more information than we might think. Several years ago a friend of mine wrote an add-in for PowerPoint that automatically added the data behind graphs (series, category, value)in a linear fashion to the alt-text of the graph. While I couldn’t read that and absorb it, we heard back from many blind users who were grateful they now had access to the data. Sadly, the add-in broke when Excel became the graphing engine for PowerPoint, but it does still work for version 2003.
The mind is an amazing thing. I suggest reading See What I’m Saying: The Extraordinary Powers of Our Five Senses by Lawrence D. Rosenblum if you truly have a commitment to creating end products that are accessible to all persons and want to be inspired to put forth the extra effort needed to create them.

This discussion is insightful but also full of misconceptions.

Go on Google books and browse Polly Edman’s book on Tactile Graphics. You will see thematic maps and line graphs designed for blind people. Also search for Susan Osterhaus’s video series on youtube, and see how blind students learn mathematics and geometry by building 2D graphs with pins and rubber bands.

On the topic of sonification, there has been a conference (ICAD) dedicated to this and related topics since 1992. That’s more than 20 years of research. I don’t know much about this research, but I would certainly feel the need to skim a few papers before I can start discussing this topic seriously.

Vision is not only for spatial representations, and non-visual senses are not only for text or speech. Sight, hearing and touch can all be used to convey either spatial information (data visualizations), numbers (data tables) or prose (the data “message”).

Vision is not purely parallel, and non-visual senses are not purely serial. We can certainly quickly spot motion and pre-attentive visual features within a wide viewing angle, but visual inspection requires eye movements. As observed in a later blog post, no one can perceive and interpret a visualization in half a second. Conversely, touch is not purely serial: no one uses his right index finger tip to search for a light switch in the dark. Audition has some degree of parallelism too: we can hear and interpret sounds coming from multiple sources at the same time.

Perception is both parallel and serial, and to paraphrase Gibson and many others, effective perception requires exploratory action: vision is naturally coupled with eye movements, head movements and occasionally locomotion. Touch is naturally coupled with motor movements. Audition is mostly passive in our everyday experience but some work in HCI has shown how it can be effectively coupled with motor movements like touch input.

Perhaps the power of visual data representations do not stem from their visual nature, but from their spatial nature. Space exists independently from vision. Vision certainly has a much higher bandwidth than all our other senses, but we might be able to achieve a lot by finding effective ways of coupling non-visual (auditory and tactile) displays with motor actions.