Physics has all the ingredients for scientific crisis

It turns out all you need is money and a googol of hypotheses to write up

In 2016, a famous survey was published in Nature called “1,500 scientists lift the lid on reproducibility.” It appeared to confirm that scientists themselves agreed there was a “significant crisis” in science.

What had been theory in the 2000s and confirmed empirically in the 2010s was now accepted by most researchers. It turned out they knew partially from their own experience that it was true and worrying. The loop had been closed, it seemed.

The “hard” fields of physics, chemistry, and engineering fared best in the poll. Now, thanks to physicists Sabine Hossenfelder and Jesper Møller Grimstrup, we may need to start rethinking what became a popular idea, that the crisis is due to the “softness” of social psychology (and, for some reason, several other fields).

Reading Hossenfelder and Grimstrup’s books Lost in Math and The Ant Mill is a delight and there’s real metascience in them. The crisis in physics is a natural experiment in which many of the other potential causes of the crisis from other fields have been removed, leaving only the pure passion for publication and funding.

In physics, you take away the questions of validity and flexible interpretation from psychology. You take away the closed data and incomplete methods from cancer biology and others. (The existential threat of not getting along with your mentors and co-authors and the other familiar pathologies of science are still there.)

This experiment suggests that there’s one common factor in science, the fact that the people who support you have no idea what you’re doing and you have little incentive to tell them.

Both authors have made an argument the public may have a chance of understanding, that hypotheses chosen from a near infinite set with nothing to recommend them have no chance of being true. Grimstrup estimates there are 10⁵⁰⁰ possible hypotheses in string theory. Each can be a paper.

It turns out, according to these very detailed accounts, that physicists will happily milk string theory just as psychologists invented new primes.

Familiar patterns

Much of the observations in these books are familiar. They have been observed in other fields and by other reformers. Goodhart’s Law, the Matthew Effect, a disturbing element of evolution in science towards worse practices.

Some observations are refreshing that you don’t often see, certainly not in psychology. The idea that researchers should be judged not only by the compromises they make for the sake of their careers, but for their silence in the face of such an obvious elephant in the room. Grimstrup even calls for — nondestructive — reform from outside. Hossenfelder has done more than her share on that front.

These are, in other words, ordinary observations from an extraordinary place few understand and few would suspect.

Physicist Alan Sokal once derided the encroachment on academia by those who deny the existence of objective reality. Years later, physicists started to deny reality too. Hossenfelder’s conversation with the eminent cosmologist George Ellis in which he laments this development, that some physicists are claiming theories don’t need to be supported by experiment, and feels powerless to do anything about it, is particularly affecting.

The most worrying part of these books is there doesn’t seem to be any natural limit before we’ve written about all 10⁵⁰⁰ hypotheses. Grimstrup paints a picture of physics most certainly continuing with groupthink fueled by government grants and the metric boosting multi-authored paper.

The power of the crowd

Hossenfelder has 1.7 million followers on YouTube and is an influential physicist in her own right. Her discussion with George Ellis was on something they’ve been saying since 2014. Hossenfelder and Ellis represent the widest influence imaginable both popular and scientific. We should be worried that the most powerful people who have worked in science, people like Walter Koroshetz, Jay Bhattacharya, Jacob Cohen, Daniel Kahneman, Csaba Szabo, Simine Vazire among many others, are often powerless to direct the crowd.

John Ioannidis is the world number 8 in citations. It doesn’t mean he gets what he wants, and published research is still theoretically, and many times now empirically, half wrong.¹ It is certainly not yet Bayesian.

The books underscore that going against the grain in science is lonely and one’s certainty starts anecdotally and personally. Grimstrup’s copies of emails from professors, editors, and peer reviewers are not as powerful as a conversation he recounts with his friend in which his friend admits he can’t take the chance on any new theories as much as he may like them. He has to play the game and look for easy publication and surer funding.

It is almost literary, to know your tribe’s ideas are wrong but you and your loved ones can’t stop pretending to believe them. In these moments, Grimstrup excels.

I spent seven years at the Niels Bohr Institute where I have been at countless seminars on string theory and I have never, not even once, heard anyone in the string theory community mention the very obvious problem that this approach has: its inability to produce falsifiable prediction and the fact that it has failed in those few occasions where it has made contact with data.

The books are a delight, and horrifying. In a way they are reassuring. The crisis has reached the basement as Hossenfelder puts it, the bedrock of science. Although it doesn’t look like there will be any end forthcoming, it is clear now that there is no field where the crisis could have stopped. The bedrock might be our only hope.

1. If this binarization of findings into “right” and “wrong” troubles you, perhaps “half of the effect when remeasured” will ease your mind.