A few days ago a data visualization developer friend of mine, Robert Monfera, sent me a link to a blog post titled “On methodological terrorism” by a thoughtful statistician named Robert Grant. Grant lays out an intelligent and entertaining case for speaking out against methodological flaws in scientific research—a practice that some on the receiving end characterize as “methodological terrorism.” He, I, and a growing number of others are speaking out to expose bad methodological practices in scientific research, not because we enjoy conflict, and certainly not because we’re assholes, but because bad science is always a waste of time and resources and it sometimes causes harm.
Grant wrote his blog post, in part, as a response to an article in a magazine of the Association for Psychological Science written by Susan Fiske, who decried the venomous nature of critiques and coined the term “methodological terrorism,” along with a few other bombastic terms, including “destructo-critics, “data police,” and “vigilante critique.” Fiske seems to be describing people such as Andrew Gelman, Ben Goldacre, Gerd Gigerenzer, and John Ioannides, whose work and integrity I greatly admire.
Here are a few of my favorite excerpts from Grant’s blog post:
If we view it as our civic duty to promote good research, it is also our civic duty not to tolerate bad research.
There is a corrupt system which you are obliged to end, and you will have to act outside the system to do so. Not by blowing up their offices…but by confronting their work when it is wrong, in the best scientific tradition, and refusing to go away until it is fixed.
Fiske [the scientist who coined the term “methodological terrorism”] said “it’s careers that are getting broken”; yes, that is precisely the objective. Acting out of ignorance, then seeing the light and fixing the problem is one thing, fighting not to change is another, and someone who refuses to learn and improve is not a scientist…
We should scare them all right, but in a thoroughly scientific way. It needs to be clear that nobody’s blunders are safe from being called out. We need to go after anyone and everyone, not just the big names.
The following excerpt from Grant’s blog post describes the circle-the-wagons resistance that the information visualization research community has exhibited in responses to my critiques:
The current system of a small number of the same people approving funding for studies, doing them, and editing the journals where they are published is arguably corrupt. The subject experts who run it benefit so much from it that they certainly don’t allow dissenting voices on their patch, and, unable to control self-publication on blogs and social media, react forcefully. Journals and conferences are used as an organ of repression, and we should focus on influencing them and not allowing them to be a refuge for irresponsible conduct.
Grant points out that researchers who cry foul in response to critiques of their work or the work of their communities tend to characterize those critiques as crossing the line into meanness. It is ironic that they often oppose these critiques through truly mean attempts at character assassination (“kill the messenger”) rather than rational discourse, which demonstrates the weakness of their position. Borrowing an analogy that was used during a keynote presentation last year by the designer Mike Monteiro, Grant likens the work of critics to that of dentists:
Now, consider the dentist. You pay them to tell it like it is. If your molar is rotten and has to come out, you want to hear it and have some straight-talking advice on what to do about it. You don’t enjoy hearing the news, but better now than later in agony. That is the service they provide — to tell you the facts, not to be your friend. We need to stop being friends of subject experts and start being their dentists instead.
Take a few minutes to read Grant’s blog post in full. If you care about science, you’ll find it worthwhile.