When we need advice in our personal lives, to whom do we turn? To someone we trust, who has our interests at heart and is wise. So why then do we often rely on advisers in our professional lives whose interests are in conflict with our own? If your work involves business intelligence, analytics, data visualization, or the like, from whom do you seek advice about products and services? If you’re like most professionals, you unwittingly seek advice from people and organizations with incentives to sell you something. You either get advice from the vendors themselves, from technology analysts with close ties to those vendors, or from journalists who are secretly compensated by those vendors. That’s not rational, so why do we do it? Usually because it’s convenient and sometimes because we don’t really care if the advice is good or not, for it is our employers, not us, who will suffer the consequences. If we actually care, however, we should do a better job of vetting our advisers.

It should be obvious that we cannot expect objectivity from the vendors themselves. Even when a vendor’s employees post advice from independent websites and claim that their opinions are their own, they remain loyal to their employers. In fact, it’s a great marketing ploy for vendors to have their employees post advice from independent sites rather than from their own. It suggests a level of objectivity that serves the vendor’s interests and multiples their presence on the web. We must also question with similar suspicion the objectivity of consultants and teachers who have built their work around a single product.

What about technology analyst groups, such as Gartner, Forrester, and TDWI, to name a few of the big guys? These organizations fail in many ways to maintain a healthy distance from the very technology vendors that are the subject of their advice. In fact, they are downright cozy with the vendors.

Trustworthy technology advisers go to great pains to maintain objectivity. They are few and far between. To be objective, I believe that advisers should do the following:

• Disclose all of their relationships with vendors. This is especially true of relationships that involve the exchange of money. If they accept money from vendors, they should willingly disclose the figures upon request.
• Do not allow vendors to advertise on their websites, in their publications, or at their events.
• Only accept payments from vendors for professional services specifically rendered to improve the vendor’s products or services. Payments for marketing advice does not qualify.
• Do not publish content prepared by vendors.

Try to find technology analysts and journalists who follow these guidelines. Even with diligent effort, you won’t find many, because there aren’t many to find.

Try an experiment. If your company subscribes to one of the big technology analyst services (Gartner, etc.), next time they produce a report that scores BI, analytics, or data visualization products, ask them for a copy of the data on which they based those scores, along with the algorithms that processed the data. This is likely done in an Excel spreadsheet, so just ask them for a copy of the file. After making the request, watch them squirm and expect creative excuses. Most likely they’ll say something along these lines: “Our scoring system is based on a sophisticated and proprietary algorithm that we cannot make public because it gives us an edge over the competition.” Bullshit. There is definitely a secret in that spreadsheet that they don’t want to share, but it is not a sophisticated algorithm.

After they refuse to show their work, move on to the following request: “Please give me a list of the vendors that you evaluated along with the amount of money that you have received from each for the last few years.” They won’t give it to you, of course, and they’ll explain that they cannot for reasons of confidentiality. Think about that for a moment. It is no doubt true that they promised to never reveal the money that changed hands between them and the vendors, but shouldn’t this clear conflict of interest be subject to scrutiny? Technology analysts and the vendors that they support are not fans of transparency.

There are a few technology advisers who do good work and do it with integrity. If you want objective and expert advice from someone who is looking out for your interests, be sure to vet your advisers with diligence and care. Question their motives. If it looks like they’re acting as an extension of vendor marketing efforts, they probably are. If, on the other hand, you’re just looking for easy answers, abandon all skepticism and do a quick Google search and then read the advice that receives top ranking. Or, better yet, schedule a call with the analyst group for whose advice you pay dearly in the form of an annual subscription.

Take care,

(Postscript: Yes, I consider myself one of the few data visualization advisers in whom you can trust.)