Patent Failure: The USPTO’s Oversized but Undernourished New Dashboard
This blog entry was written by Bryan Pierce of Perceptual Edge.
The United States Patent and Trademark Office (USPTO) recently posted an online dashboard to allow the public to monitor data related to the issuance of new patents. At Perceptual Edge, we strongly support efforts to make government data more accessible and understandable. Unfortunately, in its current form, the Patents Dashboard fails to do this.
When I first viewed the Patents Dashboard, the first thing that struck me was its size. As you can see, the dashboard is about seven screen-lengths long!
(To view the full-sized dashboard, click the image. If the newly loaded image is still small, click it to make it full-sized.)
One of the principles of good dashboard design is that you should try to put all of the information that people might want to connect and compare onto a single screen. If people have to scroll to see everything or navigate between different screens, it makes it almost impossible to make comparisons and see relationships between the variables.
There are occasional cases when a dashboard must be split between more than one screen because there is so much information that must be viewed that it cannot fit on a single screen, even when viewed at a high level of aggregation. This would be a rare case and it’s definitely not the case here. Although the dashboard takes up 7 screen-lengths, there are only 17 quantitative measures that are being displayed here. This low data density is caused by three main factors. First, most of the data is being displayed using circular gauges which take up a lot of space but only tell us a single number (for instance, we can see that First Office Action Pendency is 26.2, but we don’t know whether that’s good or bad, how good or bad it is, or how it compares with a target, last year’s value, or some other comparative measure). Second, there are large sections of text that are used to explain what each of the variables mean and how to interpret them. Generally information like this should be kept off the dashboard by putting it in a help screen or by using tooltips that appear when people hover their mouse over a particular metric. However, because many of the people who use this dashboard will be first-time users (as opposed to a more typical corporate dashboard that might be used by the same person each day), having descriptions on this dashboard isn’t necessarily a problem, they just need to be handled better. Third, this dashboard takes up so much space because the gauges are repeated. Each gauge appears once at the top of the dashboard and then again in the detailed section. This might be necessary when the dashboard is seven screen-lengths long, but if the dashboard had been created using the kind of space-saving display mechanisms that good dashboard design requires, it wouldn’t have been necessary.
Below is a redesign of the Patents Dashboard that I mocked up using space-saving means of display like bullet graphs and sparklines. I encourage you to click the image to view it at full size as the image below is too small to be seen clearly:
As you can see, by using bullet graphs and sparklines, rather than circular gauges, I was able to fit all of the metrics from the original dashboard onto a single screen and I still have about 25% of the screen left over for additional content. I also included a couple of line graphs to facilitate easy comparisons between the number of patent applications that are received and the number that are processed. In an attempt to make the information easier to digest, I’ve arranged the metrics into logical groupings so that people can easily see the different types of data and make meaningful comparisons between related variables.
Rather than displaying descriptions next to each of the metrics, I’ve designed it so that people can simply click on a metric and read its description in the box at the bottom of the screen. In my mock up, the “Traditional Total Pendency” metric has been clicked, so it’s highlighted with a light gray border and its description appears in the bottom box. This design makes it easy for people to get descriptions right on the dashboard, but it doesn’t clutter it by displaying all of the descriptions all the time.
There’s one additional thing I want to point out. It is common to see a large title, a logo, and decorative graphics at the top of dashboards. In the USPTO example, the top 1/3 of the first page is filled with these things. The top portion of the dashboard, especially the top-left, is the most visually weighty real estate on the computer screen. It should be filled with important data, not a silly image of a person looking at bar graphs through binoculars.
If you’re creating a dashboard for external users and need to include branding, consider placing it in the bottom-right corner. This is the least prominent portion of the dashboard, so it’s perfect for non-data content, such as logos, which must be included, but shouldn’t dominate.
I built this redesign based on the assumption that the metrics the USPTO chose for their dashboard are useful to people who work with patents. Given that I don’t work in the field, I chose not to create additional metrics to fill in the empty space. However, the USPTO could surely find a good use for the free space, either by including other useful metrics, or by including other graphs to give more context to the existing metrics.
My simple redesign took a seven screen-length dashboard and collapsed it to ¾ of a screen (assuming a screen resolution of 1024 x 768), while adding additional information. I was able to do this because I used display mechanisms that were designed with the tight space requirements of a dashboard in mind. Because the design uses sparklines instead of full line graphs for the time-series information, we can’t make magnitude comparisons or determine individual values within the lines, but we can see the pattern of change through time and spot exceptions, and we can do so for every variable, not just the nine that included time-series information in the original. Using the dashboard we can quickly get an overview of the data and spot metrics with interesting values, patterns, trends, or exceptions. Once we find something interesting, we could then drill into the metric to see a different screen with more detailed graphs. The purpose of a well-designed dashboard is not to try to display all the details that might eventually be needed; it’s to display the most important information at a level that people can quickly comprehend, so they know where to begin if they need to dig in for more information.
-Bryan
29 Comments on “Patent Failure: The USPTO’s Oversized but Undernourished New Dashboard”
I think the fundamental mistake they have made is by not starting from the user. “What does the users take away?” (in the first minute or so when we have their full attention) should be the starting point rather than “How can I show all that I got?”.
Definitely agree with the general idea behind this blog entry.
However, when it comes to the use of sparklines for trends, I don’t get it.
It is nice, for sure, but it does not give me any interesting information. Am I alone to have that impression? I mean, if I need information about a trend, I would rather prefer a chart with axis, labels, dates, numbers which I could analyze and draw conclusion from. You will tell me, for sure, that this information is condensed in the sparklines, which is there to give a vague idea of the trend, but still I am not convinced it gives me either a clear view on the evolution of the performance of the USPTO, or even a relevant first glance at it.
Note that the bullet chart, however, completely fulfills its task, i.e. focusing on one value compared to its target.
So my question would be: “What conclusion may I draw from any of these trend sparklines?”
Brilliant redesign, absolutely brilliant. I like how you’ve highlighted actual vs target which was not evident from glancing at those gauges (backlog is a great example). Admittedly I started to think, ooh they’ve exceeded their target, isn’t that a good thing? and then I realised.. though the design facilitated a very quick recovery from that error, and i’m not sure how it could be prevented. Small price to pay for being able to see all the pertinent information at a glance though. I imagine it takes much longer to determine if the metric you are after is even represented in the original – comparing backlog, was one of the first things i noticed on your design, and completely missed it on the original!
My one word review of The United States Patent and Trademark Office’s (USPTO) recently posted an online dashboard is:
Ugh!
@dmgerbino
John,
You’re right, because they don’t have labeled axes, sparklines provide significantly less information than line graphs. However, if I had used 17 line graphs in my design, rather than 17 sparklines, it would have taken significantly more space. The main things you can see by glancing at one of these sparklines are whether the metric is trending upward, downward, or remaining relatively flat, whether it is exhibiting some sort of pattern, and whether there are exceptions that might be interesting, such as the spike near the beginning of the Pendency of RCEs sparkline. We can’t tell anything about the magnitude of change, but we can see enough about the trends, patterns, and exceptions so that we can know where to drill down to see a full-sized line graph. Sparklines aren’t as informative as line graphs, but they provide significantly more information than trend arrows, which are what you’ll often find in use on the tight real estate of a dashboard.
Also, there are alternative sparkline designs which I could have used that provide a bit more context. For instance, here is an example of sparklines that display the highest and lowest values, which provides us with a rough sense of scale.
These sparklines require more space that they would if we didn’t label the high and low values, but if we want to give people some sense of magnitude, without using full-sized line graphs, they can be a useful variation.
-Bryan
Mark,
You’ve highlighted a potential issue with bullet graphs, for which there is no perfect solution. In one of his previous blog entries, titled “Bullet Graphs for Not-to-Exceed Targets,” Stephen discussed some of the possible ways to address this issue. In my design, I reversed the qualitative shading in the background of my “not-to-exceed†graphs, but I could have also addressed it in one of the other ways that Stephen discusses.
-Bryan
Bryan,
Thank you for the explanation.
Using trend sparklines to discover interesting patterns, variations and/or exceptions is useful. As you mentioned, you also need a way to drill down the data. Access to both a global and a detailed view is necessary to draw conclusion, and this is often an issue with the dashbards I receive on a day-to-day basis.
There is often a huge gap between the global dashboard I receive, and the spreadsheets flooded with dozens of detailed tables. Condensed information on top, very detailed information at the bottom, but rarely anything to fill the gap. And I personaly think decisions would be better and faster if based on reports emerging from this gap to be filled.
-John
Hi Bryan,
Good article – thanks.
What software did you use for the mock-up?
Matt
Hi Matt,
I used Adobe Illustrator for my mock-up, but a functional version of my design could be made in any good dashboard tool, including Corda CenterView, which is what the USPTO used for their design.
-Bryan
Thanks Bryan
Hi Byran,
You have done a great redesign of the USPTO dashboard in this mock up. I love it how the information is immediately clearer and doesnt feel over-whelming to me at first glance. Thanks for also sharing some of the thoughts that you have in redesigning and I am pretty sure we all can go further with interactive elements like drill down and tool-tips, yet, it is not an excuse for anyone to be producing sloppy designs.
Cheers
http://www.altosoft.com
Obviously, the redesign is significantly better than the original – not saying much, of course. It does, however, suffer from a significant flaw – for a single-use user, it’s almost completely unreadable.
Those of us who are readers of this blog are obviously familiar with the techniques used, so we’re able to glance at the dashboard and immediately understand what we’re seeing (aside from the not-to-be-exceeded issue, detailed in previous replies); Should a user without the appropriate background look at this, however, they would almost certainly be overwhelmed. Learning how to read multiple uncommon chart types (bullet, sparkline) for what is likely to be a single use strikes me as an unreasonable demand from our potential users, for example. Another significant issue us the relative lack of value derived from the sparkline in its current form, which may turn it into little more than clutter for most users.
Assuming the target is indeed a dashboard that can easily be used by anyone, without training (as suggested in the original post), I think a few changes might be in order:
1) Replace the gray shades of the bullet charts with light yellow and light red – people are used to traffic-light colors, so they will easily be able to see whether a value is good or bad (this can also be done by coloring the text value, displaying a traffic-light gauge, or similar means). I’m not a fan of introducing extra color into dashboards, but in this case, I feel it’s merited, and using a muted shade of these colors won’t create too much clutter.
2) Provide scale and/or target bands for the sparklines – as mentioned in an above reply, scales can be added with min/max values. Bands showing the target values (same as the grays used in the bullet charts) both give a working scale, and add information that’s immediately obvious even to a first-time user, without losing the trend-analysis value of the sparkline. For example, by adding the target bands for the top variable, we would see that the significant downward (positive) trend displayed in the sparkline is about 1/6 of the total required to go from the starting location to the target (and behind pace for a 5-year plan).
3) Moving the “Click on a variable for definition” text to the upper left, or to another more visible location, would likely make it easier to find – currently, a user might well be overwhelmed by the graphs before their eye ever catches the text in the lower part of the screen.
4) The redesign does not appear to provide a way to drill into more detail, unlike the original, which has large and immediately obvious link to detailed information. Since we’re using the variable names as links to variable information, this has to be added somewhere else in an obvious way.
One thing that we sometimes seem to forget – and which my users regularly like to remind me of – is that dashboards should, where possible, be intuitive enough that anyone can understand them with a glance. I think in this case, a few changes might facilitate easier immediate use, and the price to pay for these changes is reasonable despite the distaste of having to use less-than-optimal dashboard design in the process.
-Or
P.S. Good to see the blog active again after a couple of quiet months. :)
I disagree with the “trafic-light colors”. In my opinion, the reds seem to attract too much attention, regardless of how muted/ light they are. The legende is on the top right corner, and I believe replacing them with red/yellow/green would improve the understanding of the dashboard only marginally at best.
As for the design, it is a great improvement and I totally agree that most(all?) readers of this blog would immediately understand it since we are familiar with the type of graphs on the dashboard.
I believe the main problem for the first time/ “unexperienced” viewer, is the bullet graph. However, they are real easy to understand and quick to explain, and the redesigned dashboard has generous place to include a quick guide to bullet graphs.
As for sparklines, I don’t believe these are difficult to understand, since they basically are small line graphs. The problem is that lack of scale, so a redesign with high and low value on the sparklines would make them immediately understand for the first time viewer.
In my opinion, the first time viewer would spend more time understanding the metrics then understanding the graphs. Excellent redesign, only real improvement I can come up with, would be adding a scale to the sparklines.
I forgot, there is one more “flaw” in the design: The bottom, second from right line graph titled “Net of Applications Received vs Processed for Previous 12 months”. The title should be more clear, since you can’t process a negative number of applications. I guess it’s the difference between received and processed that are ploted, not received vs processed (which would be a scatterplot).
Or,
I appreciate your comments and the essential thrust of your argument. We should be cautious, however, in our pursuit of “intuitive†data displays. If by intuitive we mean that someone without any introduction to the display and training in its use can understand it with a glance, as you say, I don’t agree that the intuitiveness of a display should take precedence over its ability to inform. While I do agree that you want to make a display that people use only once as intuitive as possible, we often err when we support intuitiveness over meaningfulness and usefulness. Had William Playfair emphasized the intuitiveness of his charts over their usefulness, he would have never invented the bar graph or used lines to display change through time, which would have left the world starving for ways to see patterns and trends in data or to compare entire sets of data rather than just two values at a time.
Most people who will derive value from the U.S. Patents Dashboard will not use it only once. For example, patent attorneys might look at this once a month or so to see how things are going. To add stoplight colors to the dashboard to make it more intuitive for one-time users would make it less useful for those who actually need the information, which is not a justifiable compromise. By providing a simple legend as Bryan did in his redesigned version of the dashboard, he made it possible for one-time users to read the bullet graphs after a few seconds worth of instruction. It is better to ask one-time users to invest a few seconds up front than to ask those for whom the dashboard was designed to wade through the visual offense of traffic light colors forever.
We should always carefully weigh the benefits of a display’s intuitiveness (that is, understandability without instruction) versus its effectiveness (that is, usefulness). A more useful design almost always trumps and less useful but intuitive design. Fortunately, in most cases we can make information displays fully intuitive and fully useful.
Stephen,
I certainly agree with most of what you write – in this case, the argument seems to boil down to who our target users are. This in turn needs to be split into several concerns:
1) One-time Vs. repeat users – this has already been addressed in the above posts, and there’s not much to add – people using our dashboard regularly would benefit from the redesign, while one-time users may need a more simplified version.
2) How invested are our users in using the dashboard? This is similar in vein to whether people will be reusing the dashboard, in the sense that an invested user is likely to spend time to understand the different displays and variables involved in our dashboard, whereas a user who is less invested may simply give up if they do not immediately understand what they see. Making a dashboard more intuitive to a first-time viewer, particularly one who is not trained in reading dashboards (I expect most patent attorneys do not regularly use dashboards for their line of business), may in this case merit decreased effectiveness.
3) What are our users looking for? One thing that stands out in both the original dashboard and the redesign is that neither makes any attempt to display our progress towards the long term goal of reducing first/total pendency to 20 months – by 2015. This issue is ignored in the original dashboard (as are targets in general) and displayed only as a static 20-month target in the redesign. This suggests more of an “Are we there yet?” set of questions, placing less emphasis on “Will we get there in time?”. For a simple “Are goals being met?” set of questions, a complex design that will not be intuitive to some users may hinder rather than help – while a simple traffic light gauge (much as we all dislike them) would serve that purpose in a manner intuitive to anyone.
I’d like to make it quite clear that I think the redesign is excellent for anyone who will take the time to learn and use it (though I still think scale/target bands for the sparklines would improve things) – I’m just not sold that its intended users would be willing to take that time for something they look at once a month or less. Failing to account for a dashboard’s intuitiveness to its target users may result in a perfectly-built dashboard presenting all of the required information effectively, which will not be used by anyone – hardly our goal. In this case, not knowing who our target users are or what their usage patterns are, there’s certainly no way to reach a clear resolution of the optimal complexity level.
Regards,
Or
Anders,
In reference to your comment about the “Net of Applications Received vs Processed for Previous 12 months” line graph, this graph displays a deviation relationship. The gray baseline at zero represents the applications received and the blue line represents the applications processed. When the blue line is above zero, it means that more applications were processed than were received, so the USPTO worked off some of their backlog. When the blue line is below zero, it means that they processed less applications than they received, so the backlog got larger.
I could have encoded this information using two regular lines (like I did with the line graph on the right). It would have made it possible to see the actual values for applications received and processed, rather than just their difference, and it would have potentially been more familiar to people. However, I felt that what was most important was to see how the number of applications received and the number of applications processed deviated, so I encoded this directly, rather than make people perform mental math on two regular lines.
-Bryan
Bryan,
Sorry for not making myself clear. I do agree with your points above, and that the chart is good and an excellent choice for this case.
My minor issue is the title. I read “Received vs. Processed” as a scatter-plot with number of received applications along the x-axis and the processed along the y-axis. We both agree that would be the wrong chart here. Also, I do believe the legend is wrong: The dark blue line isn’t processed applications; it’s the difference between received and processed. And the light blue line isn’t received applications; it’s the 0-line (and maybe a target line, if your goal is to process all the received applications, which then should be a light grey line).
It might be nit-picking, but I do feel it’s important that the title describe the chart as accurate as possible. My suggestion is to rephrase the heading (don’t use “vs.â€) and match the legend with what’s plotted.
Anders
The original digital dashboard shown is huge! Many times a large performance dashboard can be compacted by using drill down.
Hi Anders,
You’re right. The legend in my original redesign was erroneous and unnecessary. While I don’t think there’s a problem with describing a deviation relationship using “vs.”, in removing the legend I was able make the graph title clearer as well. I’ve updated my design in the original blog post to reflect these changes.
-Bryan
Stephen,
Congratulation on this fine analysis of a Dash Board, form, fit and function.
As an implementor of corporate change, the Dash Board helps.
Had Tony Hayward had one, his “Laser Focus on Safety,” his would have been Flashing red on Deep Water Horizon actions as required to post by Quality Team on site; concerning “Change Process” and “safety.” It would have “prevented” America’s Chynoble.
Where can I buy one and use it for my clients.
Jim
Just found this web site after I had purchased an eBook on Excel dashboards and hit a URL that was referenced. I was particularly interested in the topic as I am currently 22 months into the application process for a personal patent. I guess the good news is that the USPTO is at least trying to keep me informed. But it’s clearly more show than substance. Which is why I love the reformatted version the Stephen put together.
As a first time observer of sparklines and bullet graphs, I may be a good candidate to comment on the intuitiveness vs. effectiveness discussion between Or and Stephen. It took me all of 13 seconds to understand the bullet graph. OK, maybe it was more like 20 seconds. Anyway, those 20 seconds were a sound investment of my time. It was easy to view the bullet charts and understand how the USPTO was performing. I imagine the use of color could be an enhancement, but I didn’t find the grey shades to be ineffective. They work just fine.
Very nice read. Now if you could only help me get my application moved along….
Brilliant!
Two remarks, though. First, 12 month graphs are very common, but extending them to the last 13 month make them even more useful by giving the monthly reading from the previous year as well for comparison. Second, when using colors (say, a traffic light encoding) keep in mind that almost a tenth of all male users have some kind of color blindness. Being one of them, just as an example, some commonly used hues of green and yellow are almost indistinguishable for me on screen or print. This makes me favor the grey tones.
Bryan,
Very well done ! Very nice redesign of the USPTO dashboard
!
http://www.bonavistasystems.com/samples/Dashboards.htm
Clicking the KPIs updates the KPI description text in the bottom.
I hope you like it
Andreas Lipphardt
Really great redesign. Just curious, what tool did you normally use to design the dashboard? Did a little searching on the site and couldn’t find recommendations.
Hi Gary,
Because the redesign is just a mock-up, I used Adobe Illustrator to create it. However, this design could be created in any dashboard tool that allows you to place graphs anywhere on the screen and supports bullet graphs and sparklines (or allows enough customization of its graphs that you could recreate them). It could even be designed in Excel using a bullet graph add-in.
Bryan
Bryan,
I like the redesign but one thing that confuses me is “Application Processed minus Applications Received”.
I can see the number of received applications but where is the number of processed applications in the original dashboards?
Thomas
Hi Thomas,
The original dashboard has a metric called “Utility, Plant, and Reissue (UPR) Patent Application Production Units” which basically represents the number of applications that have been processed. For my redesign, I thought it made sense to simplify the name to “Applications Processed” and present it in a way that allows people to easily compare it with the number of applications that have been received.
Bryan
Hi Bryan,
thanks for the explanation. Looking at the data again, I see what you did, nicely done.
Thomas