No YOU'RE a slug
Holden Thorp's war of words
In the past month, the reproducibility crisis has become a Republican thing. Trump's “Gold standard” executive order spurred the “Stand Up for Science” letter signed by Brian Nosek and others. Although the signatories agreed up through section 6 of the order, and Brian said section 3 is all the things his mission is to advance, the letter’s objection is too much control in the hands of political appointees. This prompted an op-ed from director of the White House Office of Science and Technology Policy in Science, setting up this response from Science editor-in-chief Holden Thorp.
All of these documents depend on the ebbing reputation of science. They will seem more or less legitimate depending on whether you think there is a replication crisis, a credibility revolution, or not much of a crisis at all because science is about “bumbling around.” Boosting any of them is unfortunately political because, as we all know, American life is a zero sum game from birth to death where your every breath must be spent extinguishing those who would cost you political power. Never cease this, even when plumbing reality itself. All must perish in these flames.
But enough about that. Holden Thorp’s editorial was, as usual, dependent on the ebbing reputation of science. What you think of the practice of science will help determine what you think of a Holden Thorp essay. If you think science is mostly okay, you might agree with him that the process of writing papers and getting peer reviewed results into journals produces knowledge so certain that making counterfactual claims about it is “not covered by free speech.” Maybe you think science is really okay if you agree with that one.
The problem is that Holden’s boy Science who has knocked over your planter is really a good boy, he swears, and yet the boy is peeking between Holden’s knees with that sneer on his face. This is just boy stuff! What he needs is discipline, and he’s going to get it when he gets home. Don’t worry.
The essay is full of replication crisis intervention language, typically “We need to encourage schools to tie promotion to good practices. We should empower early-career researchers.” Journals, like the one Dr. Thorp heads, “need to be held to higher standards.” And so on. All of these things are true and yet they don’t seem to work. I agree with COPE guidelines and teaching the replication crisis in school and my opinion that their implementation in reality is complete and utter horseshit should reinforce that. Don’t pretend to care about the words in a declaration if you don’t care about the reality they entail.
Thorp takes this wordiness to an extreme. Why, it wasn’t my boy. It’s this sluggishness and defensiveness that has taken over the town. It’s those bad apples creating a “steady stream of problematic images.” That is what has led to this “so-called ‘replication crisis.’” (For anyone unfamiliar, saying it’s “so-called” is a cheeky way to dispute twenty years of science in a few keystrokes.)
It began, he says, “mostly in psychology,” in underpowered studies, and it “created the impression that unreliable research is widespread and not reproducible.” He goes on to say that there are many errors in Alzheimer’s research as well.
How the crisis began in psychology, was not widespread and yet leapt to Alzheimer's research is not explained. Maybe it was carried on this “steady stream” of problematic images connecting the two events in this verbal playground.
Dr. Thorp is skipping over cancer biology, lots of psychology, economics, cancer biology again, ecology, nutrition science, and hundreds of papers covering every aspect of the scientific process and its ebbing reputation. In 2016, most researchers agreed there’s a replication crisis off the top of their heads.
“The scientific community needs to face up to the extent to which its own actions have fed this perspective.” Perception. Perspective. These are the worst parents. The “don't get caught” parents.
“Although science is driven by data, politics and public opinion are shaped by anecdotes and storytelling. Thus, pointing out that many of these incidents are the result of a small number of actors does nothing to change the political narrative.”
[Emphasis added.]
The replication rate and the many practices we know with mathematical certainty are driving those numbers lower are not practiced by a small number of actors. One of the reasons we know the “incidents” are not isolated to psychology and then cancer biology and then coincidentally economics and recently Brazil is that they all have the same practices that are, very simply, caused by hypothesis testing for the purpose of publication. These practices are p-hacking, base-rate neglect, and the proverbial granddaddy, the one we should never, ever go to a publisher for an opinion on, publication bias.
The number of actors involved in publication bias is not small. It’s very large and Holden Thorp doesn't get to say what we do about it.
The essay goes on. Dr. Thorp deserves some credit for saying that the issue lies in the general area of his boy. If you work in science and you’re “sluggish” or “defensive,” your ears are really burning.
He also deserves some credit for saying, “the scientific community should be engaging in a conversation about problems and potential solutions.” Although “conversation” is another favorite of essay writers and bad parents, I think this is good. Let’s converse, and let’s include everyone. Let’s do this quickly for god’s sake. The boy’s in college.