Perceptual Edge’s 2012 Dashboard Design Competition

A few weeks ago I mentioned in this blog that I would soon announce the 2012 Perceptual Edge Dashboard Design Competition. Today, the competition officially begins. This will be the most challenging event of this type to date resulting in the most esteemed award for dashboard design (in my not-so-humble opinion) since I judged a similar competition for the B-Eye-Network back in 2006. This competition will serve several purposes:

  1. A showcase for the current state of expert dashboard design.
  2. An opportunity for me to use the submissions to teach best practices by critiquing several of them on this website and in the second edition of the book Information Dashboard Design, which I am currently writing.
  3. An opportunity to provide sample dashboard designs that could actually be used to improve the quality of education in schools, for this competition involves the design of a dashboard that could be used by teachers to monitor the performance of their students.

The winning dashboard will be featured in Information Dashboard Design, Second Edition, due out during the first half of 2013, and in an article in the Visual Business Intelligence Newsletter. No, you won’t win $10,000 or an all-expenses-paid vacation to the Bahamas. Instead, you will have an opportunity to advance the information age by showing a better way to display data for performance monitoring. In other words, you will have a chance to do something useful for the world.

Here are the basic facts:

  • The deadline for submissions is September 28, 2012.
  • Submissions should be emailed to info@perceptualedge.com.
  • Multiple entries from an individual are allowed.
  • Submissions will be judged by me—Stephen Few—with the assistance of two subject matter experts in student performance monitoring. The identities of the competitors will be hidden from me until after the winners have been selected.
  • Anyone may enter the competition, including employees of vendors that sell dashboard software.
  • You may use any software that you wish to design this dashboard. In fact, you are welcome to design a dashboard using drawing software if you’d like, such as Adobe Illustrator, which would free you from the constraints of a particular dashboard tool. The purpose of this competition is to assess your dashboard design skills, not the merits of a particular tool.
  • The student performance data that will be used is provided in an Excel file, which you may download be clicking here. The file contains behavior, aptitude, and achievement information for students in a single high school mathematics class.
  • By submitting a dashboard to the competition you are granting me the right to include it my courses and publications.
  • The winner will be chosen based on one fundamental criterion: the degree to which the dashboard could be used by a teacher to rapidly and effectively monitor the performance of her students for the purpose of helping them improve their mathematics skills.

All other information that you’ll need to participate in the competition is included in the Excel file mentioned above.

I encourage you to enter the competition and to take it seriously. The world needs better ways of monitoring information, and you can help by applying your skills to this task. Whether you win the competition or not might matter to you, but it isn’t the only thing that matters. Either way, you will learn a great deal through the process.

Take care,

60 Comments on “Perceptual Edge’s 2012 Dashboard Design Competition”


By Andrew Fox. August 2nd, 2012 at 1:10 pm

This looks a fun challenge. To give us some more direction will you be assessing the entries against the standards you advocate in your books or what the non acedimic corporate world consider is a Dashboard ?

Andrew Fox

By Stephen Few. August 2nd, 2012 at 1:36 pm

Hi Andrew,

I’d prefer to answer your question in this way: I’ll be evaluating the merits of the dashboards based on their ability to be used for effective and efficient performance monitoring. By this I mean that a typical high school mathematics teacher, with a little training, could use it to quickly and clearly understand how her students are performing based on the metrics that I’ve provided. This is neither an academic perspective nor a corporate perspective; it is a usability perspective. (By the way, I’ve spent a great deal more time working in the corporate world than in the academic world.) Because I wrote my book Information Dashboard Design to teach best practices for designing performance monitoring displays that really work, then it is true the criteria that I will use to evaluate these dashboards are in line with the content of my books and courses.

Make sense?

By Andrew Fox. August 2nd, 2012 at 1:56 pm

It makes sense that you would want to evaulate any entries in line with the content of your books.

A few years ago I promised you at a course in London a dashboard using SAP Dashboards (Xcelsius) to critique, so count me in.

By Stephen Few. August 2nd, 2012 at 2:00 pm

Andrew,

Wonderful. I still haven’t ever seen a well-designed dashboard in Xcelsius, but I’d be thrilled to see one and will happily give it its due.

By Coy Yonce. August 2nd, 2012 at 2:24 pm

I’ve been looking forward to this since you announced. I haven’t yet decided upon a tool. Any chance you would accept multiple entries from an individual?

Looking forward to seeing what you produce Andrew. I’ve seen some of your other work with Xcelsius and you know it well.

By Stephen Few. August 2nd, 2012 at 3:26 pm

Coy,

Yes, multiple entries from an individual are allowed. I’ll update the announcement to reflect this fact.

By Mike Crow. August 2nd, 2012 at 6:56 pm

Hi Stephen,

I appreciate the interesting challenge, in a topic that’s been on my mind. I hope to submit a dashboard. I have a couple of questions about the details:

1. What is the score range for each letter grade?

2. Regarding the grade goal, was it created by the student, the teacher, or both?

Cheers, Mike

By Stephen Few. August 2nd, 2012 at 8:25 pm

Hi Mike,

The grade goals were created by the students. Regarding the score ranges for the letter grades, why does it matter?

By Pieter Hendrikx. August 2nd, 2012 at 11:24 pm

Hi Stephen,

I really enjoyed your 3-day workshop in the Netherlands a couple of months ago. You can count me in on this competition.

I already see some interesting names in the comments from the SAP community :)

Best regards,
Pieter

By Josh Tapley. August 3rd, 2012 at 8:45 am

Yes, Pieter… there does appear to be a lot of us!

When I heard about this, I was thinking about an HTML5 dashboard, but now I guess I’ll have to submit with Xcelsius too so I don’t feel left out.

Best of luck everyone,

Josh Tapley

By Tim2. August 4th, 2012 at 4:46 pm

“By submitting a dashboard to the competition you are granting me the right to include it my courses and publications.”

I’m hoping not as an example of bad dashboard design.

By John. August 5th, 2012 at 9:52 am

“I still haven’t ever seen a well-designed dashboard in Xcelsius, but I’d be thrilled to see one and will happily give it its due”

“Ever”? http://sfew.websitetoolbox.com/post/show_single_post?pid=22540126&postcount=25

To be fair, I can’t claim to have “designed” this in Xcelsius, as much as I can say it was “created in Xcelsius, based on Stephen Few’s design”. It also required many workarounds to get it to look like that, but Xcelsius has come a long way since then (although admittedly, I have seen the light in Tableau - way easier).

Nevertheless, I’ll be participating in this as well. It should be fun and I’m looking forward to see what others create.

John (Mindjon)

By Stephen Few. August 5th, 2012 at 9:57 am

My apologies, John. I had forgotten about this. I will amend my statement. I have seen only one Xcelsius dashboard that I consider to be well designed.

By Stephen Few. August 5th, 2012 at 5:26 pm

Tim,

I will likely include some of the dashboards that are submitted as examples that can be improved and will critique them as such, similar to way that I critiqued submissions to the sales dashboard competition that I previously judged in the first edition of Information Dashboard Design. I share the hope that your submission will not fall into this category.

By Dimitar Dimitrov. August 6th, 2012 at 1:41 am

Does the submitted design have to depict precisely the provided data, or is it acceptable the images to have imprecisions as long as the design is implementable with the specified data model? This is especially important for designs done in general purpose drawing programs.

By Stephen Few. August 6th, 2012 at 9:29 am

Dimitar,

If you use a drawing tool, don’t worry about representing values precisely in the charts. For our purposes, approximate values are acceptable.

By John Doe. August 7th, 2012 at 6:51 am

Is it ok to submit a result of group effort rather than that of individual?

By Stephen Few. August 7th, 2012 at 9:21 am

Mr. Doe,

Groups efforts are certainly acceptable and might even be fun.

By Guy Cuthbert. August 7th, 2012 at 1:21 pm

Stephen

Could you explain what form a submission should take? Are you looking for sample image(s) e.g. JPG/PNG? Is a PDF acceptable?

Apologies if you have covered this elsewhere in the competition rules - I did search them first!

Thanks

By Stephen Few. August 7th, 2012 at 1:27 pm

Hi Guy,

Submission requirements are explained in the Excel file that contains the data. Any image file will do (JPG, PNG, BMP, TIFF, etc), as well as a PDF.

By Becky. August 12th, 2012 at 9:54 am

Hi Stephen,

I understand that you are looking for designs of an overview dashboard (no drilling down or tooltips) and will only be judging visual design, not the function of any elements. I also understand that the submission will be a staic image file.
However, can the design include snapshots of some dynamic elements (i.e. a filter on a chart) where the functionality would be noted in the accompanying document description?

Thanks!

By Stephen Few. August 12th, 2012 at 10:01 am

Hi Becky,

Yes, you may submit snapshots to illustrate dynamic elements, such as filters. Keep in mind, however, that the purpose of the dashboard is rapid performance monitoring, so dynamic elements should support that task, not something unrelated to performance monitoring.

By Scott Reida. August 13th, 2012 at 11:31 am

I think that using a program other than Excel wouldn’t be easy to implement on a large scale in the environment we’re looking at. We have enough trouble keeping someone trained in DOMO Centerview (our program of choice…or force). Teachers are going to quit before they start if we throw new programs at them. Anybody else want to weigh in on this…and have experience for/against my claim?

By Andrew. August 14th, 2012 at 9:12 am

@Scott:

A person does not need training in order to use a dashboard. If you design the dashboard effectively, which is the purpose of this competition, then whether in Excel, CenterView, or even as a screenshot (as Stephen requested), anyone familiar with the subject matter (in this case, student performance) should be able to make sense of it without any training.

As for CenterView, it displays dashboards through a web browser, so there isn’t any need to “throw new programs” at teachers; they should be more familiar with using a web browser than with using Excel.

By Stephen Few. August 14th, 2012 at 4:52 pm

Andrew,

While I agree with the spirit of your comment–that a dashboard should be simple to use–it is not true that training is never required to use one. Training is definitely will be required in some cases, and that’s fine. For example, would you refrain from including bullet graphs on a dashboard if the users have not already been trained to read them? In this case it would be worth the 60 seconds of training that’s required to use a means of display that is superior to already familiar alternatives. Training that is required to use a dashboard should be minimal and should not be required when a better design could have eliminated the need.

By Andrew. August 15th, 2012 at 10:59 am

@Stephen:

Fair enough. I was responding to the idea of conventional training (of the expensive/extensive variety) as implied by Scott’s dilemma: “keeping someone trained”. Such training should rarely be necessary to use a dashboard, but I did forget there are many other forms of training.

For example, I generally try to supplement my dashboards with a short document explaining any unfamiliar metrics or means of display. This form of training mitigates the issue of people forgetting their training, as they can simply refer to the document. It also eliminates the need for me to talk to people.

By John Doe. August 17th, 2012 at 3:51 am

Is it ok to guess and present missing data, such as exact date of exams, to plot them along with absent/tardy data (to show possible correlations)?

By John Doe. August 17th, 2012 at 4:46 am

I’m not familiar with US school system. Could you please explain more about assignment 1~5 scores?

By John Doe. August 17th, 2012 at 5:29 am

Since it’s Algebra I class, I thought current grade is 9th. But sheet says ‘My Other 10th Grade Classes.’ So it looks like current grade is 10th. Is it 9th or 10th?

By Stephen Few. August 17th, 2012 at 8:38 am

John,

You may fabricate and present missing data if you wish. Regarding the assignment 1 through 5 scores, so far during the school term students have completed five course assignments in order from 1 through 5. Scores are expressed as percentages out of 100%. Students in this class are in the 10th grade.

By David Hoppe. August 20th, 2012 at 5:28 am

Hi Stephen,

Just as John Doe, it’s not clear to me what the assignment scores are referring to. Is it to the term, this grade, or somethig else? Can you please clarify these data?

Thanks, David

By Stephen Few. August 20th, 2012 at 7:55 am

David,

The assignment scores refer to homework assignments and exams that were given so far during the current school term (i.e., in the 10th grade).

By Brad Earle. August 24th, 2012 at 10:58 am

Stephen, I, like Mike, would like to have some more background info regarding this teacher’s approach to grading. The critical element of how the grades are to be calculated for the course is typically part of the syllabus provided by the teacher at the beginning of the course. In order to figure out whether a student could achieve their desired goal given that 5 assignments have already been completed is dependent on how many assignments are proposed for the class, any weighting that is done on assignments and the teacher’s proposed score to letter grade scale. Having this information would enable a Teacher to have early awareness that a student’s goals are either 1. achievable with adjustments (or not) to a student’s current performance trend or 2. not achievable at all, given past performance.

Given that the Ms. Metcalf’s objective is a dashboard that leads her to constructive conversation/action with particular students, it would seem that knowing where a student stands based on what she announced would be the grading approach for the class is a crucial piece of missing information.

By Stephen Few. August 25th, 2012 at 9:53 am

Brad,

Excellent questions. This is the kind of thinking that’s required to develop effective performance monitoring displays.

A total of eight assignments will be given in the class. To keep things simple, let’s assume that the assignments carry equal weight for determining the overall course grade. Letter grades are assigned as follows: 90-100% is an A, 80-89% is a B, 70-79% is a C, 60-69% is a D, and everything below is an F.

By Xavier. August 28th, 2012 at 12:24 pm

Hi Stephen,

Interesting questions (and answers) in the comments of this post, I must say.

I have a question also that has not been discussed yet: what is the frequency of the Algebra 1 class? Daily? Based on the distribution of student absences and tardies dates, I assume that Algebra 1 is a daily class. Is that correct?

Thank you.

By Stephen Few. August 28th, 2012 at 6:22 pm

Hi Xavier,

Yes, this class meets daily.

By John Doe. August 30th, 2012 at 3:06 pm

Stephen,

The data shows that “disciplinary referrals >= detention count” holds for all cases. Are detentions caused by disciplinary referral? Or are they independent?

Could you explain more about the relationship between them?

Thank you.

By John Doe. August 30th, 2012 at 3:42 pm

Stephen,

(In regards to Brad’s question and your answer to that):

1. Is the Previous Course Grade means last term or previous grade(9th grade)?

2. Can I assume that the Current Grade determined entirely by the assignment scores? (FYI, each student’s current grade matches with the average of assignment scores.)

Thank you.

By Stephen Few. August 31st, 2012 at 5:30 pm

John,

It would be unusual for a detention to occur without being preceded by a disciplinary referral. A disciplinary referral occurs when the teacher sends the student to a member of the administration for discipline. During the meeting with the administrator, if the student’s behavior warrants severe discipline, a detention is the result.

Assume that the Previous Math Course Grade refers to the math course that the student took during the previous school term. Also, assume that the current grade is determined by the average of the student’s assignment scores.

By Ediuislei. September 3rd, 2012 at 2:48 pm

Ahmed,The most important elmneet of the design is of course the spreadsheet model. The look and feel / the eye candy / the funk comes later so time spent on the spreadsheet is critical. However there is nothing special about the data movement etc. I use a lot of Vlookups() (shouldn’t really as they are fat functions) and I use a lot of hidden combo boxes you can use those to move data around a model. It’s a clever trick because the default’ selection of a combo can be a cell. So if you use some control to set a cell, a hidden combo can pick that value up, set itself to that value and it’s FILTERED ROW capability can then move data without the user touching the combo box.Actually just look at this example:You do not need to use the combo box, just the number wheel and you’ll that you can move data round a model using any control you want / need. You can get the source here:The XLF is included and shows how this works.RegardsDaniel

By Stephen Few. September 3rd, 2012 at 3:13 pm

Ediuislei,

Beware. The integrity of the underlying model is critical, but not more so than the effectiveness of the presentation. Neither can be compromised.

By Rob. September 5th, 2012 at 9:10 pm

Stephen,

I wonder if you could provide some background on what the assessment scores represent. Do the 6th Grade thru 9th Grade assessment scores represent the students’ final course grade in those previous years? If so, I would expect the previous math course grade to correspond with the 9th grade assessment score, but that doesn’t appear to be the case (eg. Nikolas Mikhailovich has 57% 9th grade assessment and an F for previous course grade). Does the latest assessment score represent something in the current course term? Are assessments exams? I’m just not familiar with this concept.

Thanks! Looking forward to working on this…can’t wait for that trip to the Bahamas, oh wait… :)

By Pete Z. September 7th, 2012 at 8:39 am

Rob,

Although Stephen has not yet responded, I can offer up my interpretation (assumption). I would think that assessment scores refer to something similar to standardized tests that are conducted in US public schools as part of the No Child Left Behind act. Here in New York state, for example, a standardized test is given once a year in math and ELA (English Language Arts). These results (supposedly) reflect the academic success of a school. In some way or another, these standardized tests may impact the school’s funding from the government. The standardized test results aren’t factored into a student’s course grade.

Again, this is my interpretation of “assessment scores” as they exist in the excel file. The challenge with assessment/standardized tests is whether a teacher will choose to “teach to the test” rather than teach to acquire mastery of the material. One would think that achieving mastery of the material would ensure success on a standardized test, but not necessarily.

By Stephen Few. September 7th, 2012 at 9:26 am

Rob,

Pete’s explanation is correct. The assessment score pertains to a standardized test, not to anything that was done in a class.

By Pete Z. September 7th, 2012 at 10:26 am

Stephen,

Quick question — Is the teacher for this class color blind?

By Stephen Few. September 7th, 2012 at 10:31 am

Pete,

Good question. Only about 0.5% of women are color blind. This teacher is not one of the few women with this particular problem.

By Tim2. September 10th, 2012 at 3:53 am

“Letter grades are assigned as follows: 90-100% is an A, 80-89% is a B, 70-79% is a C, 60-69% is a D, and everything below is an F”

Quick question, are scores rounded when grade boundaries are considered? For example, if somebody averaged 79.6% would that be seen as less than the B boundary so a C grade or rounded to an integer so a mark of 80% which would be a B?

By Stephen Few. September 10th, 2012 at 4:25 am

Tim2,

Scores are not rounded to determine letter grades.

By Francis. September 10th, 2012 at 7:57 pm

Is it correct to assume that the “latest standardized math assessment score” is for the 10th grade, even though the school year is not over? The other tests have a grade attached to it (6-9) so I’m wondering why this one is not identified as that of the 10th grade. Many thanks.

By Stephen Few. September 10th, 2012 at 10:50 pm

Francis,

Yes, the latest standardized math assessment was given to these students during the current 10th grade school term. The standardized test is not identified as the 10th grade assessment because it is not specific to the 10th grade.

By Roman Pyzh. September 13th, 2012 at 12:41 pm

How much screen real estate (pixels, resolution) can the dashboard take up? Or is the expectation that this dashboard should be printable as a one-page document?

By Stephen Few. September 13th, 2012 at 3:49 pm

Roman,

Excellent question. Assume an SXGA screen resolution of 1280 x 1024.

By Scott Reida. September 24th, 2012 at 8:05 am

@ Andrew
WOW! I need to start this…need to get my procrastinator self into gear. In regards to the training comment I put up here a few weeks ago…I wasn’t referring to training needed to read a dashboard. I was referring to the training needed to make the dashboard itself. I assume the teacher would be the one making the dashboard in whatever program was chosen…therefore…I think that any program other than Excel should be considered a stretch for the target customer. Perhaps I understood this wrong…but the teacher is the one that would make the dashboard…
OK…four days left…

By Stephen Few. September 24th, 2012 at 8:13 am

Scott,

Rarely would a teacher be in a position to create a dashboard for herself. An effective dashboard cannot be created without skills that few teachers will ever be trained in. Dashboard design isn’t what they do. They teach. No matter how good the software, designing a good performance monitoring display, just like effective data analysis, will require skills that people like you and I must bring to the table.

By Bhumik Panchal. September 24th, 2012 at 4:24 pm

Hi Stephen,

Has the date of final submission changed from 21st to 28th September? The original data file had the 21-09-2012 as the deadline. Is there a reason for the change? Can we then update our initial submissions?

Cheers,
Bhumik

By Stephen Few. September 25th, 2012 at 7:40 am

Bhumik,

Yes the date has changed and yes you may update your submission.

By Scott Reida. September 25th, 2012 at 8:00 am

@ Stephen (and all interested),

It opens things up quite a bit if there aren’t limitations on the software. My vision for this would be that an Excel file would be distributed…with multiple tabs…one for the data entry…the rest would be fixed. On one of the fixed tabs would be the dashboard that would automatically update based on the data entry tab. Of course…adding/dropping of students/assignments/etc. would be a constant pain since the plot ranges would constantly change…and that would require a fixed page to require changes. I can’t win!

As for the deadline…I’m happy it was moved. I didn’t realize that I missed the initial deadline. Get busy everyone!

Best regards, Scott

By Matt. October 2nd, 2012 at 4:49 pm

Now that all entries have been submitted, could you share how many are we to compete ?

By Bryan Pierce. October 2nd, 2012 at 8:56 pm

Hi Matt,

We received a total of 91 entries, which is significantly more than Stephen or I anticipated, given the time and effort involved in designing a dashboard. I want to thank everyone who participated.

Stephen will certainly have his work cut out for him judging them all!

-Bryan

By Jason. October 25th, 2012 at 3:45 am

WIll you be sharing the results. I am very keen to see the entries but sadly missed the opportunity to submit one myself.

Thanks

J