<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd" xmlns:googleplay="http://www.google.com/schemas/play-podcasts/1.0"><channel><title><![CDATA[The Red Team of Science]]></title><description><![CDATA[The Red Team of Science]]></description><link>https://www.redteamofscience.com</link><image><url>https://substackcdn.com/image/fetch/$s_!NMSg!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F492515cb-cdbe-40a8-bdc8-b0495365ad50_1209x1209.png</url><title>The Red Team of Science</title><link>https://www.redteamofscience.com</link></image><generator>Substack</generator><lastBuildDate>Sun, 22 Nov 2026 13:08:17 GMT</lastBuildDate><atom:link href="https://www.redteamofscience.com/feed" rel="self" type="application/rss+xml"/><copyright><![CDATA[Alex Byrnes]]></copyright><language><![CDATA[en]]></language><webMaster><![CDATA[redteamofscience@substack.com]]></webMaster><itunes:owner><itunes:email><![CDATA[redteamofscience@substack.com]]></itunes:email><itunes:name><![CDATA[Alex Byrnes]]></itunes:name></itunes:owner><itunes:author><![CDATA[Alex Byrnes]]></itunes:author><googleplay:owner><![CDATA[redteamofscience@substack.com]]></googleplay:owner><googleplay:email><![CDATA[redteamofscience@substack.com]]></googleplay:email><googleplay:author><![CDATA[Alex Byrnes]]></googleplay:author><itunes:block><![CDATA[Yes]]></itunes:block><item><title><![CDATA[Drugs from our universe]]></title><description><![CDATA[How the FDA and a seemingly obscure equation could change science]]></description><link>https://www.redteamofscience.com/p/drugs-from-our-universe</link><guid isPermaLink="false">https://www.redteamofscience.com/p/drugs-from-our-universe</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Tue, 17 Nov 2026 13:18:36 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!poM-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Imagine a universe where a random protein out of billions in your gut doesn&#8217;t cure depression. Do you find that universe easy to imagine?</p><div class="poll-embed" data-attrs="{&quot;id&quot;:1785084}" data-component-name="PollToDOM"></div><p>If your response was that it&#8217;s easy to imagine because let&#8217;s face it, we probably live in that universe, congratulations, you&#8217;re a Bayesian.</p><p>If you&#8217;d asked the same question twenty years ago about the depression gene, one of tens of thousands of gene variants, the answer would be the same and you could have predicted the fall of many millions of dollars in research with a simple thought experiment.</p><p>The fragility of this kind of research has to do with the fact that most experiments involve assuming a universe where something doesn&#8217;t work (a drug, a gut protein etc.) and seeing how surprising the finding would be in that universe. The part that&#8217;s neglected is translating from that universe to our own and it involves a number, which we&#8217;ll call B.</p><p>Bayesian reasoning is powerful. It&#8217;s used by major pharmaceutical companies internally, yet it isn&#8217;t often used to filter obvious baloney, or fast track promising treatments at the federal level. That might be changing as the <a href="https://www.fda.gov/news-events/press-announcements/fda-issues-guidance-modernizing-statistical-methods-clinical-trials">FDA has released updated guidance</a> on its use this year.</p><p>To get an idea of how the FDA wants to use Bayesian reasoning, suppose a drug works in adults and the FDA is considering approving it in children. Statistically, you can take the previous effect size of the drug in adults, discount it in some way to account for the differences between adults and children and use that number (B) to directly calculate future success.</p><h2>Why now?</h2><p>Bayesian methods have never achieved organic growth. In the middle of the last century, the methods that became dominant were ones that experts preferred. We know this because experts borrowed parts from existing methods in a way that made publishing papers easier, and that version of the method proliferated without statisticians explicitly endorsing it. In fact, the method that dominated has been criticized for decades and continues to be used in the vast majority of research.</p><p>The method can be understood with a toy example. (This example unfortunately has a villain in it, which is the way <a href="https://doi.org/10.1016/0010-0277(89)90023-1">humans understand math</a>.)</p><p>Suppose an expert wants to report a finding but doesn&#8217;t really want to give away how likely it is to be true. If C is how likely it is to be true, an expert could simply report A where A &#215; B = C and you would never know what C is unless you&#8217;re also an expert who could provide the missing number B.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!poM-!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!poM-!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg 424w, https://substackcdn.com/image/fetch/$s_!poM-!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg 848w, https://substackcdn.com/image/fetch/$s_!poM-!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!poM-!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!poM-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg" width="1179" height="786" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:786,&quot;width&quot;:1179,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:283753,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/225228223?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!poM-!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg 424w, https://substackcdn.com/image/fetch/$s_!poM-!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg 848w, https://substackcdn.com/image/fetch/$s_!poM-!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!poM-!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2f571e9f-0b8b-43df-8494-7093b2aec7bf_1179x786.jpeg 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>It can&#8217;t be that simple, though, right? No, of course not. We can&#8217;t just regulate the disclosure of B and force researchers to give up a number. There&#8217;s a simple proof of this that comes from Wall Street. Investors spend billions of dollars trying to get the number from experts in the field from different drug trials. The pharmaceutical company GlaxoSmithKline elicits B from its experts by getting them to estimate all at once, averaging their estimates together. Even then, it&#8217;s not so easy. Facilitators are told to try to help experts to give their real estimate, not the one they would <a href="https://doi.org/10.1002/pst.1854">like to be true</a>.</p><p>This might be what happens at FDA and it&#8217;s a real concern. Bayesians have been fighting with Frequentists for a century and this is one area where the Frequentists have a point. At the edge of human knowledge you can&#8217;t verify expert opinion without becoming one yourself, particularly when it comes to divulging a number. Simultaneous confession by experts working at the same pharmaceutical company might be the best we can do.</p><p>So, trying to force experts to formalize what they know might not better inform the public. However, in an age where the general public <a href="https://wellcome.org/insights/reports/wellcome-global-monitor-2026?trk=feed_main-feed-card_feed-article-content">distrusts experts</a>, this methodology is hard to ignore. It's particularly hard when there are cases where B is not revealed by experts but by common sense or counting. (One in a billion proteins in your gut really is a probability of one in a billion.) There&#8217;s an obvious information asymmetry between experts and the public that, at an aggregate level, has been maintained instead of broken down. Surely some form of asking for the number couldn&#8217;t hurt.</p><p></p>]]></content:encoded></item><item><title><![CDATA[Are academic scientists honest?]]></title><description><![CDATA[Let&#8217;s ask a scientist.]]></description><link>https://www.redteamofscience.com/p/are-academic-scientists-honest</link><guid isPermaLink="false">https://www.redteamofscience.com/p/are-academic-scientists-honest</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Wed, 11 Nov 2026 12:03:56 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!OIW2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>A small thing happened about six weeks ago. An academic reported a single finding at a relatively obscure conference, one of thousands that day in a sea of millions of papers this year. The difference is that this result is special. It affects all the others: <em>Are academic scientists honest?</em></p><p>If you&#8217;re familiar with the way that academic scientists study other academic scientists these days, some of what follows will seem normal but I&#8217;ll say it anyway. She reported three findings on a slide. On the third one, the direction was left off. The finding was that academic and non-academic researchers <em>differ</em> in their integrity according to her poll.</p><p>I asked anonymously in which direction they differed. Did academics or non-academics have more integrity? My question was ignored throughout the program. They even checked off the other questions to show which ones were answered.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!q3i9!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!q3i9!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png 424w, https://substackcdn.com/image/fetch/$s_!q3i9!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png 848w, https://substackcdn.com/image/fetch/$s_!q3i9!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png 1272w, https://substackcdn.com/image/fetch/$s_!q3i9!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!q3i9!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png" width="1456" height="910" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:910,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:312953,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/223895780?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!q3i9!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png 424w, https://substackcdn.com/image/fetch/$s_!q3i9!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png 848w, https://substackcdn.com/image/fetch/$s_!q3i9!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png 1272w, https://substackcdn.com/image/fetch/$s_!q3i9!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46fc3469-c29c-4723-930c-f7efd5f2996c_1920x1200.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The next day, the long wait for the conference videos began. First they were sent so you couldn&#8217;t see them if you&#8217;re not at University of Helsinki. Then, after repeated requests, most of the videos were uploaded but this wasn&#8217;t one of them. Then the rest were promised last week.</p><p>Little did I know, the variable wasn&#8217;t reported anywhere else, but it hardly mattered. The full study had been released in an <a href="https://beyondbadapples.eu/news/behavioral-insights-for-improving-research-integrity/">obscure PDF</a> on the BEYOND project&#8217;s web site and it is such a cornucopia of academic researchers behaving badly and being covered by others that it renders the answer moot.</p><h2>The poll</h2><p>The BEYOND project polled 634 researchers with scenarios like, &#8220;A funder asks you to change your results&#8230;&#8221; For each scenario, respondents have four options that they&#8217;re asked to agree or disagree with. Comply and delete the findings, don&#8217;t delete them, et cetera.</p><p>In each scenario, the participants reliably disagreed with the worst option. However, the things they said they would do instead are quite shocking and, true to the pattern I&#8217;m trying to establish, the BEYOND project papered over how bad it looks by coding all but the worst behavior as &#8220;high integrity.&#8221;</p><p>In one scenario, a finding depends on removing a control variable from the analysis, a classic no no. A significant number of respondents said they would remove the variable and &#8220;look for scientific arguments for doing so&#8221; afterward. This was called a &#8220;high integrity&#8221; activity in the report.</p><p>Another scenario asks what you would do if you found a mistake in your own published paper. A significant number of people agreed with the option to discuss the issue with a supervisor and follow their advice regardless of what it was. In fact, there was significant agreement in three places in the poll to privately tell other researchers about errors and simply let it go depending on what they say.</p><p>Other &#8220;high integrity&#8221; options are worthy of Machiavelli:</p><p>1. Doing some alternative analyses, finding they don&#8217;t support the hypothesis, and then listing &#8220;reasons why performing these additional analyses would be inappropriate&#8221; in the paper.</p><p>2. Revising a paper at an editor&#8217;s request such that the hypothesis is based on a different explanation after the fact.</p><p>3. Arguing with reviewers about including variables you secretly know aren&#8217;t significant.</p><p>As bad as these are, responses hovered around &#8220;neither agree nor disagree&#8221; on average.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!SQxy!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!SQxy!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png 424w, https://substackcdn.com/image/fetch/$s_!SQxy!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png 848w, https://substackcdn.com/image/fetch/$s_!SQxy!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png 1272w, https://substackcdn.com/image/fetch/$s_!SQxy!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!SQxy!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png" width="728" height="1029.5" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:false,&quot;imageSize&quot;:&quot;normal&quot;,&quot;height&quot;:2059,&quot;width&quot;:1456,&quot;resizeWidth&quot;:728,&quot;bytes&quot;:1043849,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/223895780?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:&quot;center&quot;,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!SQxy!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png 424w, https://substackcdn.com/image/fetch/$s_!SQxy!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png 848w, https://substackcdn.com/image/fetch/$s_!SQxy!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png 1272w, https://substackcdn.com/image/fetch/$s_!SQxy!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F088c52c6-19ee-47b6-81f2-96602aa18e4f_2481x3508.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">The scenarios. About one square from the center line is significant if all groups are past the line even with maximum correction (Bonferroni, 40 comparisons) and assuming maximum variance |t| &#8805; 4.19, p &#8804; 0.0013. (BEYOND project deliverable 3.2, Figure 3)</figcaption></figure></div><p>In almost every case, the most sacrificial answer, the one that the rest of society would call high integrity was not popular, taking one&#8217;s name off a paper, submitting to another journal, or attempting to publish a non-significant result.</p><p>These results shouldn&#8217;t surprise anyone who knows how the sausage is made and it shouldn&#8217;t have surprised the BEYOND project. In 2026, a study from the same EU funder found that only about 35% of researchers think p-hacking is unacceptable of which all of this is some form, from improper to extremely improper. 42% found <a href="https://doi.org/10.1038/s41598-026-39928-z">manipulating data in a project proposal</a> acceptable, one of the few of these practices that is, broadly speaking, illegal.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!OIW2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!OIW2!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png 424w, https://substackcdn.com/image/fetch/$s_!OIW2!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png 848w, https://substackcdn.com/image/fetch/$s_!OIW2!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png 1272w, https://substackcdn.com/image/fetch/$s_!OIW2!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!OIW2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png" width="1456" height="911" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/ef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:911,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:2749640,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/223895780?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!OIW2!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png 424w, https://substackcdn.com/image/fetch/$s_!OIW2!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png 848w, https://substackcdn.com/image/fetch/$s_!OIW2!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png 1272w, https://substackcdn.com/image/fetch/$s_!OIW2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fef31ac96-4fdb-4d9f-8059-499ec6786649_1585x992.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The BEYOND project, which is short for &#8220;beyond bad apples,&#8221; as if they knew what was coming, could have written a detailed preregistration predicting that the poll would turn out this way. Why didn&#8217;t they do that and end up being spectacularly right? Because this experiment was not intended to measure integrity. It measured whether integrity could be improved by... priming, one of the most ludicrous and thoroughly rejected hypotheses in all of science. No surprise, career-ending integrity was not improved by paragraphs of text.</p><p>If you really want a laugh, read how this experiment was described at the <a href="https://www.wcri2015.org/downloads/former-conferences/9th-wcri-2026-vancouver/proceedings-6/839-abstract-book-final/file">World Conference on Research Integrity</a>, keeping in mind that the only positive result they found was on the scenario where they specifically asked participants to think about the scenario in different ways. It&#8217;s not even priming. They got a significant difference when they asked participants directly to produce a difference, and none otherwise.</p><p>That&#8217;s right. Researchers presenting at a scientific integrity conference studying how negative studies are spun into positive ones spun their predictably negative result into a positive one while papering over real integrity concerns revealed by their own <em>direct measurement of integrity concerns</em>. This was then papered over again by another conference on research integrity.</p><p>It&#8217;s worse because if most of the &#8220;high integrity&#8221; activities aren&#8217;t actually high integrity, then their other findings could easily reverse. For instance, they reported that people with more published papers have higher integrity.</p><p>All of this surprising behavior is explained by a simple fact: We greatly underestimate how strong incentives in science are. The incentives described in the scenarios are hell. They truly are. If you can imagine them being real and routine, then you might not need a scientist. You can answer the question yourself.</p><p></p>]]></content:encoded></item><item><title><![CDATA[A review of Google’s AI paper checker]]></title><description><![CDATA[Deep Review is impressive, flawed, and historic]]></description><link>https://www.redteamofscience.com/p/a-review-of-googles-ai-paper-checker</link><guid isPermaLink="false">https://www.redteamofscience.com/p/a-review-of-googles-ai-paper-checker</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Mon, 19 Oct 2026 12:35:42 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!oZzr!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Last week Google released Deep Review, an AI-generated peer review platform. This sort of thing is right up my street, but that equally may make me biased.<br><br>Deep Review is a practical, nondenominational peer review platform that specializes in long, complex documents. Like many Google products, it&#8217;s safe, simple, and seemingly unopinionated. If any company is going to capture the risk-averse university and large institution market for AI peer review, it&#8217;s probably Google.</p><p>I think having a boring, standard paper reviewer is a good thing. Google may bring people to the table on pre-publication and post-publication review.</p><p>The flip side is of course the usual dangers of centralization and that users may consider Google Deep Review, or anyone&#8217;s AI, the automatic truth. On balance, I&#8217;m in favor of making things easier for laypeople and getting more straightforward errors out of science and so a free paper checker backed by a trusted company seems positive. I <em>can&#8217;t possibly know how fair or rigorous Deep Review is or will be</em> but so far, so good.<br></p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!oZzr!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!oZzr!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png 424w, https://substackcdn.com/image/fetch/$s_!oZzr!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png 848w, https://substackcdn.com/image/fetch/$s_!oZzr!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png 1272w, https://substackcdn.com/image/fetch/$s_!oZzr!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!oZzr!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png" width="1446" height="1087" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/f72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1087,&quot;width&quot;:1446,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1152751,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/220856744?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!oZzr!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png 424w, https://substackcdn.com/image/fetch/$s_!oZzr!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png 848w, https://substackcdn.com/image/fetch/$s_!oZzr!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png 1272w, https://substackcdn.com/image/fetch/$s_!oZzr!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff72c8e36-cd2a-4215-adc2-7607d1cfb291_1446x1087.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>I uploaded two papers that I&#8217;d already reviewed publicly that predate the release of Deep Review. <a href="https://www.nber.org/papers/w33697">Alcott et al.</a>, the viral Facebook-deactivation-makes-you-happy paper, and <a href="https://doi.org/10.1038/s41586-026-10992-9">Schwaba et al.</a>, the association between genes and personality featured in <a href="https://www.nytimes.com/2026/09/03/science/genes-personality-traits.html">The New York Times</a>.<br><br>Both of these are important papers for the kind of democratization of scientific public opinion that I&#8217;m interested in. Insiders and statisticians should have no trouble critiquing and then ignoring these papers, but the public, journalists, and influential twitter personalities may take them at face value. On this, Google Deep Review and other LLM platforms have a chance to do some good and there&#8217;s evidence that they will. Both reviews came up with some of the same points I raised. They are points some have wanted science and the public to understand for a long time.</p><h2><br>Facebook deactivation</h2><p><br>Last year, a working paper achieved over 5 million views thanks to a glowing tweet storm calling deactivating Instagram or Facebook comparable to therapy. I took it up as a chance to do a lesson plan for students learning critical appraisal in April 2025.<br><br>Deep Review found 7 major and 23 minor errors in the paper. For the most part, I found the minor errors ignorable, things like spelling or punctuation. However, it called this error minor:</p><div class="pullquote"><p>&#8220;...the effect is &#8216;statistically distinguishable from zero ... at the (p=0.14) level,&#8217; before noting in the next sentence that it does not meet the pre-registered (p=0.05) significance threshold.&#8221;</p></div><p>My <a href="https://www.redteamofscience.com/p/a-students-guide-to-post-publication">review</a> had this as question number 1 on the quiz. The authors did diverge from their promise to use a p &lt; 0.05 threshold, and there is no &#8220;0.14 level.&#8221; This is not a minor error because it&#8217;s misleading. It&#8217;s easy for insiders to pick out, hard for outsiders.</p><h5><br>Payments may be the thing making participants happier</h5><p><br>This is one of the main problems with the paper. People who were happier also got more money. The effect size is small and therefore could easily be explained by a little extra money. Participants could find $125 more in a week a lot of fun, certainly a little bit. People working for online surveys tend to be that way.</p><p>$125 could be a birthday present, a trip to the shore, or a fancy dinner for two. This obvious flaw is not discussed. It&#8217;s a major problem, and Deep Review nailed it. It put it in somewhat bland terms, &#8220;discussing why a $125 payment differential is unlikely to drive the observed emotional state improvements would bolster the identification argument,&#8221; but it performed beautifully.</p><p>This confound is not a deep confound. One of the reasons this paper is a good first paper for students is that its flaws are fairly obvious, but boy is this kind of convergence a sight for sore eyes. I highly recommend it with humans or AI.</p><p>Another major flaw in the study is its use of subgroup analyses that were not in the preregistration. There are lots of potential subgroups to explore. The paper highlights some findings and not others. Deep Review flagged the statistical flexibility several times and aptly.</p><p>I&#8217;m sharing Deep Review&#8217;s <a href="https://github.com/alexbyrnes/google_deep_review_comparison/blob/main/Allcott_et_al_2025_review.md">output</a>, and a <a href="https://github.com/alexbyrnes/google_deep_review_comparison/blob/main/Allcott_et_al_2025_review_convergence.tsv">summary</a> of where I missed something that Deep Review found, and vice versa written by ChatGPT.</p><p>I highlighted measurement validity, lack of blinding, and practical significance that Deep Review did not. Deep Review found a lot of issues I missed including a major reporting error where the authors reversed which psychological improvement (15% or 22%) went with which platform.</p><p>Unsurprisingly, I use the lens of human motivation more and the LLM almost always sticks to errors that are self-evident and contradictory within the paper, although there&#8217;s no reason it can&#8217;t do both. Deep Review&#8217;s responses are obviously tailored to authors rather than readers. It would be fascinating to see the system prompt, and it would be fascinating to see how a product aimed at consumers of research would differ.</p><p></p><h2>Genes and personality</h2><p>The headline was &#8220;Scientists Find More Than 1,000 Genetic Variants Linked to Personality&#8221; and the premise was that we now have the technology to find these kinds of things. (This is false. We now have the power to find something interesting that is inherently buried in billions of <a href="https://www.redteamofscience.com/p/throwing-spaghetti-at-the-wall">false alarms</a>.)</p><p>Deep Review found 12 major and 38 minor <a href="https://github.com/alexbyrnes/google_deep_review_comparison/blob/main/Schwaba_et_al_2026_review.md">issues</a> with the paper. It didn&#8217;t go deeply into p-hacking and selective reporting, but it did find a multiplicity issue I did not. As before, Deep Review went into much greater detail than I did. Full comparison <a href="https://github.com/alexbyrnes/google_deep_review_comparison/blob/main/Schwaba_et_al_2026_review_convergence.tsv">here</a>.</p><p>Worryingly, one of Deep Review&#8217;s major findings is wrong:</p><div class="pullquote"><p><br>&#8220;In Extended Data Fig. 9, 29 of the 33 reported pairings achieve significance solely through weighted median and weighted mode while MR-CAUSE spans zero (with MR-CAUSE excluding zero in only 4 pairings...&#8221;</p></div><p><em>Almost none</em> of the MR-CAUSE intervals include zero. Interestingly, it missed an actual issue in the same column, two findings in bold indicating significance where the interval did cross zero. This may have been an error in character recognition since the figure was given as an image in the PDF, but Deep Review found the precise opposite of what&#8217;s in the table, which may indicate a bad assumption or miscommunication between agents.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!hlFB!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!hlFB!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png 424w, https://substackcdn.com/image/fetch/$s_!hlFB!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png 848w, https://substackcdn.com/image/fetch/$s_!hlFB!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png 1272w, https://substackcdn.com/image/fetch/$s_!hlFB!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!hlFB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png" width="744" height="723" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:723,&quot;width&quot;:744,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:310668,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/220856744?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!hlFB!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png 424w, https://substackcdn.com/image/fetch/$s_!hlFB!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png 848w, https://substackcdn.com/image/fetch/$s_!hlFB!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png 1272w, https://substackcdn.com/image/fetch/$s_!hlFB!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a15b5df-d4b8-498b-8fe3-b8f277eb9368_744x723.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption"><strong>Reproduced portion of Extended Data Fig. 9 (Mendelian randomization analyses)</strong> from <em><a href="https://doi.org/10.1038/s41586-026-10992-9">Robust inference and correlates from genetic associations with personality</a></em> (Schwaba et al., 2026)</figcaption></figure></div><p>This type of error is more dangerous for critics than authors. Presumably an author would check, or they would be familiar enough with the table not to run with Deep Review&#8217;s critique. So this error, while major, is relatively fine.</p><p>Now you might say, &#8220;You&#8217;re letting AI off the hook!&#8221; Yes. This table alone has two reversed intervals, an impossible interval, breaks its own inclusion rules, and the entire premise of the paper is false. AI outperformed 138 authors, The New York Times, along with countless readers and commenters.</p><h2>Overall comments</h2><p>General-purpose LLMs are starting to do post-publication review competently. A year ago, my experiments with LLM review were much less successful. LLMs needed hand-holding, didn&#8217;t know their own limitations, and gave vague hand wavy responses. The fact that they&#8217;ve improved substantially is heartening because I think more people need access to this information to make sense of their lives and the world around them. Peer reviewers, journalists, and authors themselves are not doing an adequate job of finding and surfacing fairly obvious and self-evident errors. Well-accepted reforms are being adopted at a glacial pace and may even reverse given the size and interests of primary research.<br><br>There are, on the other hand, major concerns with this development. As LLMs start to outstrip humans in certain activities &#8212; one of which is the ability to produce giant walls of dense text &#8212; our ability to vet them is diminished.</p><p>I&#8217;ve worked on labeling for AI companies and that concern has certainly arrived. On one job, I concluded that the only way a human could be doing the work thoroughly, reading 10 * 50,000 word outputs in an 8 hour shift is to cheat, pasting the outputs into another LLM, which is strictly prohibited. Humans simply cannot read a half million words of dense technical text in a day and understand whether or not the LLM is, for instance, lying.<br><br>The humans training LLMs may be contingent workers hired by a third party labeling company with no interest in the outcome other than getting their tasks approved. In most cases, labelers don&#8217;t even know who the client is. In the past, humans could readily identify six-fingered hands all day. Now the work is exhausting.<br><br>Deep Review may also be sycophantic towards authors. At up to $20 for a 40 page document, it ought to be. It may avoid critiquing things authors can&#8217;t control by the time they&#8217;re writing the paper like study design. Market forces may put Google in the &#8220;author services&#8221; industry, an important role but one rife with conflicts of interest. &#8220;Prepare this for Nature editor X&#8221; comes to mind.<br><br>So although my experience with these tools has only become more positive over the last six months, hurdles remain. Pitting one LLM against another, hand checking, forcing re-evaluation, and trying to maintain one&#8217;s own disinterestedness are as useful now as ever. However, there is plenty among the self-evident and easy-to-check results to say that we&#8217;re watching history being written. That much is beyond doubt.<br></p>]]></content:encoded></item><item><title><![CDATA[AI’s success is also a landmark for public critique]]></title><description><![CDATA[The democratization of scientific review]]></description><link>https://www.redteamofscience.com/p/ais-success-is-also-a-landmark-for</link><guid isPermaLink="false">https://www.redteamofscience.com/p/ais-success-is-also-a-landmark-for</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Fri, 09 Oct 2026 12:40:51 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!3utS!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>On the occasion of AI overtaking humans in math, and those <a href="https://www.youtube.com/watch?app=desktop&amp;v=1KitV-zjy18">humans admitting it</a>, it may be useful to reflect on what else that means. It means AI has already overtaken us in the simpler math found in most research papers and it means the public has access to how poor that math often is. It&#8217;s now conceivable that an ordinary person could debate the greatest scientists and it could cost those scientists their reputation.</p><p>In a world where reputation is not mechanized and mechanically inflated, this wouldn&#8217;t matter too much. Maybe AI would bruise a few egos and life would continue as usual. But we don&#8217;t live in that world. We live in a world where reputation can be hacked.</p><p>Reputation in science is thought of as reputation amongst peers transferred to the masses, but direct public scrutiny can be decisive. A claim that <a href="https://psycnet.apa.org/doi/10.1037/a0021524">ESP is real</a> helped touch off the replication crisis, not because other researchers doubted it &#8212; they privately doubt a lot of papers &#8212; but because the public did too. The methods used in that paper were routine and still are.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!3utS!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!3utS!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg 424w, https://substackcdn.com/image/fetch/$s_!3utS!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg 848w, https://substackcdn.com/image/fetch/$s_!3utS!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!3utS!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!3utS!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg" width="960" height="581" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:581,&quot;width&quot;:960,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:118123,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/219479865?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8d416676-1266-46f3-b15e-0b22569dde24_960x720.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!3utS!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg 424w, https://substackcdn.com/image/fetch/$s_!3utS!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg 848w, https://substackcdn.com/image/fetch/$s_!3utS!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!3utS!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F11cba5ad-f3ff-42e5-9461-a726aaf95f5c_960x581.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Protesters in Hungary (2025). Photo: Wikimedia Commons, licensed under Creative Commons Attribution-ShareAlike (CC BY-SA).</figcaption></figure></div><p>So it may be no coincidence that what the public doesn&#8217;t understand, research has done little to reform. That would be bad enough, but consider how easy it would have been to prepare for the day that the public has a PhD in their pocket. In the case of the most prevalent and least understood problem, a disclaimer would have cost nothing but a little embarrassment: &#8220;This paper&#8217;s result is contingent on a <em>p</em>-value. <em>P</em>-values should not be interpreted as the probability that any hypothesis is true. Additionally, without a detailed preregistration, <em>p</em>-values may have been selected for publishability rather than plausibility.&#8221; An approach with a similar goal was proposed by prominent reformers in 2012, the &#8220;<a href="https://dx.doi.org/10.2139/ssrn.2160588">21 word solution</a>.&#8221; It went nowhere.</p><h2>The last possible moment</h2><p>Some mathematicians have been forthright, saying <a href="https://www.youtube.com/watch?app=desktop&amp;v=1KitV-zjy18">AI&#8217;s superiority</a> in math has become &#8220;impossible to deny,&#8221; that early resistance was wishful thinking, and that AI has &#8220;gone from mediocre PhD student to very good PhD student to experienced mathematician level to top mathematician level.&#8221;</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!zb12!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!zb12!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg 424w, https://substackcdn.com/image/fetch/$s_!zb12!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg 848w, https://substackcdn.com/image/fetch/$s_!zb12!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!zb12!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!zb12!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg" width="1280" height="720" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:720,&quot;width&quot;:1280,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:262250,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/219479865?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3ae533dd-8e25-4e9b-9fd4-6f46d4e913fa_1280x720.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!zb12!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg 424w, https://substackcdn.com/image/fetch/$s_!zb12!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg 848w, https://substackcdn.com/image/fetch/$s_!zb12!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!zb12!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F3419ddec-6b97-4ef3-bf31-74d429e7556c_1280x720.jpeg 1456w" sizes="100vw"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Thumbnail from Sky News &#8220;<a href="https://www.youtube.com/watch?app=desktop&amp;v=1KitV-zjy18">The world&#8217;s best mathematicians already overtaken by AI</a>&#8221; (September 22, 2026).</figcaption></figure></div><p>Almost all at once, anyone can use AI for simple validation of a scientific news article or paper. Much of the time, simple checks are all it takes to push papers over. Alzheimer&#8217;s was pushed over by <a href="https://www.science.org/content/article/potential-fabrication-research-images-threatens-key-theory-alzheimers-disease">duplicate images</a>. Social science can be pushed over just by running the code and comparing the output to the claim in the paper (<a href="https://doi.org/10.1038/s41586-026-10203-5">25%</a> of it). Many papers can be invalidated by running <a href="https://doi.org/10.3758/s13428-015-0664-2">two</a> <a href="https://doi.org/10.1177/1948550616673876">scripts</a> that were available pre-AI. Many can be shown to be improbable before the experiment <a href="https://doi.org/10.1371/journal.pmed.1004085">takes place</a>. The list goes on, from <a href="https://doi.org/10.1186/s41073-025-00173-z">citation</a> errors to <a href="https://doi.org/10.1007/s11192-021-04140-5">plagiarism</a> (18%) to <a href="https://doi.org/10.1371/journal.pone.0005738">outright</a> <a href="https://metaror.org/article/approximately-1-in-7-scientific-papers-are-fake/">fraud</a>.</p><p>In 2024, a paper claiming black spatulas cause cancer was debunked by a human. Then someone decided to see if AI could have found the same error. <a href="https://secondthoughts.ai/p/the-black-spatula-project">It did</a>, and that was back when LLMs couldn&#8217;t do math well.</p><p>Of course, armchair debunking is not all that is going to happen. Researchers will run papers through AI themselves. They will get help writing papers and peer reviewing them. Many papers will be inoculated to start with. On the other hand, more papers will be written as the value of <a href="https://giorgio.gilest.ro/the-weimar-of-knowledge/">each paper drops</a>.</p><p>With any luck, though, we&#8217;ll get everything scientific reform has been asking for since 2011. At the last possible moment, but nonetheless.</p><p>If the AI then kills us, it really will be the absolute last possible moment. Those of us wearing convincing paperclip costumes will be left to write this down as the quintessential case of procrastination.</p>]]></content:encoded></item><item><title><![CDATA[Throwing spaghetti at the wall]]></title><description><![CDATA[A familiar genetic finding makes it into The New York Times]]></description><link>https://www.redteamofscience.com/p/throwing-spaghetti-at-the-wall</link><guid isPermaLink="false">https://www.redteamofscience.com/p/throwing-spaghetti-at-the-wall</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Thu, 01 Oct 2026 12:49:16 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!qfVh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>As the throughput of scientific instruments has risen over the last twenty years, the discipline required of researchers has risen too. If someone sees a publishable result flash across their computer screen, they have to admit it, along with all the unpublishable numbers they saw too. Science requires counting the spaghetti on the wall and the spaghetti on the floor.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!qfVh!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!qfVh!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png 424w, https://substackcdn.com/image/fetch/$s_!qfVh!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png 848w, https://substackcdn.com/image/fetch/$s_!qfVh!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png 1272w, https://substackcdn.com/image/fetch/$s_!qfVh!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!qfVh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png" width="1448" height="1086" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1086,&quot;width&quot;:1448,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1135204,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/218243383?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!qfVh!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png 424w, https://substackcdn.com/image/fetch/$s_!qfVh!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png 848w, https://substackcdn.com/image/fetch/$s_!qfVh!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png 1272w, https://substackcdn.com/image/fetch/$s_!qfVh!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c56cc8b-8c59-4dee-9463-a345f245f97f_1448x1086.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Arguably, the conditions that led some to conclude that half of published research is false in the early 2000s are more acute today. The spaghetti-throwing machines are too powerful. We can tell they are because, in the textbook case of genomics, a statistical guardrail was put in place so that researchers could throw no more than a specific number of spaghetti strands at the wall, and that number was 1 million.</p><p>What happens if you throw 1 million-and-one strands, or 2 million, or 20 million? The answer is that you can get published anyway. From there, <a href="https://www.nytimes.com/2026/09/03/science/genes-personality-traits.html">The New York Times</a>.</p><p>The <a href="https://doi.org/10.1038/s41586-026-10992-9">study</a> in question is one of many, but it&#8217;s particularly illustrative because it combines every statistical dimension that can be abused: a giant search space of what to test, a great number of tests actually performed, and tiny associations treated as important.</p><h1>The spaghetti tower</h1><p>Schwaba et al. has 138 authors and 16 categories of analysis. All of the analyses have some level of undisclosed flexibility and we are left to imagine how many more numbers flashed across all those screens. How much spaghetti is on the floor is hard to detect. Luckily, there&#8217;s a limit to how obvious you can be about these things and this paper barely waves from the window as it shoots through that limit.</p><p>The authors tacitly admitted they were testing beyond what the publishing threshold balances out, writing in the peer review file that they counted only the 5 analyses they would need to test the Big Five personality traits. &#8220;We use a p-value of .01, which corrects within test dividing .05/5 for the Big Five. We did not correct p-values otherwise.&#8221; In this analysis alone there were 90 tests, not 5. The full database, called Add Health, has hundreds more variables available that could have been tested too. Across the laptops and high-performance computing clusters of 138 authors, untold numbers of tests could have been run, and any novel, coherent finding could have been its own paper with failed associations dropped.</p><p>The situation with genomics today is not, as <a href="https://x.com/StevenSalzberg1/status/2095928050290413813">some in the community believe</a>, that the low threshold covers anything researchers would like to do. The threshold is static, the 1 million tests, while the number of tests run is flexible.</p><p>This may explain why the paper highlights seemingly disparate associations like the one between neurotic genomes and &#8220;having pulled a gun or knife on someone,&#8221; and a less &#8220;attractive personality,&#8221; but not with the many, many variables we would also expect to be associated with such neurotic tendencies.</p><p>We can see some lack of coherence even in the variables the authors admitted to testing, but didn&#8217;t discuss: Neuroticism is associated with the participant&#8217;s father having been imprisoned, and pulling a gun or knife on someone, but not with lying to one&#8217;s parents, drunk driving, arrest, or the participant being imprisoned themselves. Neuroticism is associated with having been suspended, but not expelled from school.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!-IL5!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!-IL5!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png 424w, https://substackcdn.com/image/fetch/$s_!-IL5!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png 848w, https://substackcdn.com/image/fetch/$s_!-IL5!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png 1272w, https://substackcdn.com/image/fetch/$s_!-IL5!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!-IL5!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png" width="1456" height="647" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:647,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:null,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:null,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:null,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!-IL5!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png 424w, https://substackcdn.com/image/fetch/$s_!-IL5!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png 848w, https://substackcdn.com/image/fetch/$s_!-IL5!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png 1272w, https://substackcdn.com/image/fetch/$s_!-IL5!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6eacd01e-082f-4e4b-9451-74eef6337db2_2048x910.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Figures <a href="https://www.nature.com/articles/s41586-026-10992-9/figures/2">2B</a> and <a href="https://www.nature.com/articles/s41586-026-10992-9/figures/3">3</a> as a distribution of results. The sharp drop-off in p-value counts is consistent with human intervention, selecting publishable values from a larger set.</figcaption></figure></div><p>The &#8220;1,000 variants linked to personality&#8221; reported by The Times is almost certainly too strong. And there&#8217;s another problem that the threshold didn&#8217;t fix: small effects. In one of the more extreme findings, people with a moderately high genetic score for neuroticism among at-risk participants still only had about 7 percentage points higher chance of having an imprisoned father than those with moderately low neurotic scores. Other effects are much smaller. The sample size meant that an association could make it into the headline even if the difference was minuscule, less than half a percentile.</p><h1>The larger point</h1><p>As it was when the math behind irreplicable &#8220;candidate gene&#8221; studies was <a href="https://doi.org/10.1371/journal.pmed.0020124">described</a> in the early 2000s, there&#8217;s a larger point than candidate genes, or even genomics. As capacity grew, more and more tests became possible, which begs for discipline that can be limiting to one&#8217;s career and to the careers of one&#8217;s collaborators. Perhaps yet more discipline is required when 138 researchers have to agree to limit their chances of getting into The New York Times, or when a single peer reviewer has to tell them they miscounted.</p><p>The success of dividing by 1 million, called the <em>Genome-Wide Significance Threshold</em>, prevented many candidate genes being erroneously associated with disease, and many, many more genes being erroneously associated as computing power and throughput rose. But it&#8217;s not magic. It was, in a way, a band-aid. The threshold could be considered one era behind even when it was implemented. Worse, the search space has expanded, making the odds that you&#8217;ve truly found something vanishingly small. The &#8220;number of gene variants&#8221; that the 1 million threshold acknowledges has become the &#8220;number of <em>sets of variants</em>&#8221; more recently, a combinatorial explosion.</p><p>The forefront of science is unpredictable. One of the signs of poor science is that it&#8217;s <em>predictable</em>. A result cannot be statistically surprising and unsurprising at the same time. Dalton Conley, cofounder of sociogenomics who was interviewed for the article, implied that discoveries like this can be both predictable and fruitful. &#8220;The next study that has double the sample size will find 3,000 variants, no surprise&#8230;&#8221; He can be sure of that not because he knows there are 2,000 more real and important variants to find this way, but because he can do the math. The authors probably could too. As it was before the first replication crisis, a discovery claim is guaranteed with a large enough sample size, enough variables to test, and ways to test them. Historically, genomics is just the field that makes it obvious.</p>]]></content:encoded></item><item><title><![CDATA[Poll finds scientific methods are adopted in order of preference]]></title><description><![CDATA[38% of professors don&#8217;t know the term replication]]></description><link>https://www.redteamofscience.com/p/poll-finds-scientific-methods-are</link><guid isPermaLink="false">https://www.redteamofscience.com/p/poll-finds-scientific-methods-are</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Wed, 26 Aug 2026 12:05:52 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!yc9K!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Most papers with bad news for academia are ignored. Once in a while a paper comes along that is destined to be ignored in its own special way. Last week, a poll found that <a href="https://doi.org/10.1098/rsos.260462">38 percent of professors</a> aren&#8217;t aware of replication.</p><p>Replication is the practice of repeating others&#8217; experiments. In philosophy it&#8217;s called intersubjective agreement and, if you think about it, it&#8217;s the bedrock of all we know. A few years ago there was a thing called the replication <em>crisis</em>. Many since have claimed this phrase is too harsh, but we&#8217;re finding out that it didn&#8217;t matter. You could have a conversation about the replication crisis around many professors and they <em>wouldn&#8217;t understand the words</em> <em>you&#8217;re saying.</em></p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.redteamofscience.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Educators can join in by requesting <a href="https://www.routledge.com/An-Introduction-to-Metascience-and-the-Critical-Appraisal-of-Research-From-First-Principles-to-Complex-Concepts/Byrnes/p/book/9781041218722">copies</a> of <em>An</em> <em>Introduction to Metascience</em>. <em>Note: The subject of this post happens to have a <a href="https://www.mheducation.co.uk/a-student-s-guide-to-open-science-using-the-replication-crisis-to-reform-psychology-9780335251162-emea-group">textbook</a> too.</em></p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><p></p><p>The paper is full of bad news:</p><ul><li><p>There isn&#8217;t a new cohort of young PhDs who have heard of open science practices at a much higher rate.</p></li><li><p>Professors were actually 2 points worse at knowing what replication is than researchers in general.</p></li><li><p>The data was a convenience sample from international reproducibility networks, so the numbers should be on the optimistic side.</p></li><li><p>The word &#8220;replication&#8221; is defined under the question. Knowing the concept would have been enough.</p></li></ul><p>The findings also show a dose-dependent response where the dose is incentives. Practices that personally benefit researchers are most known and most used. Open access is very popular and it has come to resemble pay-to-play where public funding is used to pay extraordinary fees to publishers. Preprints are second most popular and they prevent you from getting scooped. The order of prevalence roughly matches the order of personal benefit:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!yc9K!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!yc9K!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png 424w, https://substackcdn.com/image/fetch/$s_!yc9K!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png 848w, https://substackcdn.com/image/fetch/$s_!yc9K!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png 1272w, https://substackcdn.com/image/fetch/$s_!yc9K!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!yc9K!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png" width="1456" height="852" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/f41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:852,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:342669,&quot;alt&quot;:&quot;&quot;,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/212723977?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" title="" srcset="https://substackcdn.com/image/fetch/$s_!yc9K!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png 424w, https://substackcdn.com/image/fetch/$s_!yc9K!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png 848w, https://substackcdn.com/image/fetch/$s_!yc9K!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png 1272w, https://substackcdn.com/image/fetch/$s_!yc9K!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff41852a6-6644-4fa3-9ef1-71419cd0eabe_1493x874.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption"><strong>Figure 4. Percentage awareness and use of open research practices.</strong> Reproduced from Pennington, C. R., et al. (2026), &#8220;Awareness and use of open research practices: an international survey of researchers across disciplines,&#8221; <em>Royal Society Open Science</em>, 13(8)</figcaption></figure></div><p>The exception is registered reports at the bottom, a practice that guarantees publication for authors regardless of outcome. Its unpopularity is an ongoing mystery.</p><p>This poll, and many results like it, is why the academia-is-actually-okay articles are so frustrating. Someone will find a bad apple and say it&#8217;s indicative of all the apples, and the other side will say bad apples are rare.</p><p>These views assume science is an apple cart heading in the right direction. An apple falls off, or a lot of apples fall off, so what. In actuality, <em>it benefits researchers more to do worse science</em>. The net direction of the apple cart may be wrong. Public <a href="https://www.pewresearch.org/short-reads/2025/10/15/growing-share-of-americans-say-the-us-higher-education-system-is-headed-in-the-wrong-direction/">polls</a> suggest it&#8217;s true. Peer reviewed results say researchers are made of the same stuff as the rest of us.</p><p>We can&#8217;t have <a href="https://royalsocietypublishing.org/rsos/article/3/9/160384/56494/The-natural-selection-of-bad-scienceThe-Natural">cheaters prosper</a>. This may sound stark and totalizing, but the next era of the replication crisis should be marked by simplicity. We still need everyone to google the terms.</p><p></p>]]></content:encoded></item><item><title><![CDATA[Psychology is not mostly fine]]></title><description><![CDATA[Astral Codex endorses justice delayed on the replication crisis]]></description><link>https://www.redteamofscience.com/p/psychology-is-not-mostly-fine</link><guid isPermaLink="false">https://www.redteamofscience.com/p/psychology-is-not-mostly-fine</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Fri, 07 Aug 2026 11:03:54 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!Cd5p!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Last week, maybe the most prominent blogger to have touched the replication crisis, and author of one of its <a href="https://slatestarcodex.com/2019/05/07/5-httlpr-a-pointed-review/">classics</a>, Scott Alexander, wrote a piece that surprised some of his audience. &#8220;<a href="https://www.astralcodexten.com/p/psychology-research-is-mostly-fine">Psychology Research Is Mostly Fine</a>&#8221; was true to its name. In the follow-up, he addressed the apparent backlash and said:</p><p>&#8220;...just like in economics, medicine, sociology, political science, or even some of the hard sciences. All of these encountered the replication crisis at different times, all of them adjusted in different ways, and they&#8217;ve all risen to the occasion in some cases and failed to confront others.&#8221;</p><p>This doesn&#8217;t really mean anything, but to the extent that it does, it&#8217;s wrong. Alexander doesn&#8217;t have anything to cite here because nearly everything about psychology and beyond says that fields have not: 1. Tested themselves at all, 2. Retested to show improvement, 3. Done much of anything to address the crisis or even admit that word <em>crisis</em>.</p><p>On psychology, he goes on, &#8220;Psychology is no different. It has flaws, but most of it consists of good ideas and trustworthy research.&#8221;</p><p>I&#8217;ll present an argument against <em>a fortiori</em> using a textbook I happened to have to read recently. Other ways of making the same argument are <a href="https://www.redteamofscience.com/p/psychologists-think-theres-a-crisis">here</a> and <a href="https://www.redteamofscience.com/p/science-reform-do-we-need-a-plan">here</a>.</p><p>Fields have not taken the first step, education, and they&#8217;re not doing well with the others. Education is a great place to start between the plentiful large-scale, roughly 50% findings, because it is so miserably universal. There is good reason to believe that schools don&#8217;t teach the replication crisis, updated methods, or even the old methods properly according to longstanding complaints that predate the crisis.</p><p>One might assume, for instance, in the field most aware of the problems, psychology, that this 2024 textbook &#8220;<a href="https://www.mheducation.com/highered/product/methods-in-behavioral-research-cozby.html?viewOption=student">Methods in Behavioral Research</a>&#8221; by Paul Cozby would <em>encounter</em> and <em>adjust</em>.</p><p>No. Instead, what we get is a speedrun of every hypothesis-testing misnomer there is. The authors encourage the exact problem that caused the replication crisis, that &#8220;Conceptual replications are even more important than exact replications in furthering our understanding of behavior,&#8221; that we gain more certainty not with independent results or more observations, but by testing a phenomenon in different ways. Critically, they endorse the granddaddy phenomenon that will cover all the retrying up: &#8220;Many rejected papers are submitted to other journals and eventually accepted for publication, but much research is never published. This is not necessarily bad; it simply means that selection processes separate high-quality research from that of lesser quality.&#8221;</p><p>They don&#8217;t mention the danger in multiple methods, p-hacking and publication bias. This doesn&#8217;t mean that the authors have a mistaken view of p-values as a mark of quality, but we can say for certain that they do have that interpretation because they say so:</p><p>&#8220;The logic of the null hypothesis is this: If we can determine that the null hypothesis is incorrect, then we accept the research hypothesis as correct. Acceptance of the research hypothesis means that the independent variable had an effect on the dependent variable.&#8221;</p><p>Even though this is more or less the classic misinterpretation of significance testing, and the worst one, let&#8217;s give them the benefit of the doubt and say they mean in dichotomous and exhaustive cases where the prior is also very high and excuse the causal language. They paint over the problem of priors by casually assuming the hypothesis is always true:</p><p>&#8220;you are most likely to obtain significant results when you have a large sample size because larger sample sizes provide better estimates of true population values.&#8221;</p><p>Not to be left out, the classic mistake that random error is the only other explanation for the effect:</p><p>&#8220;Statistical inference begins with a statement of the null hypothesis and a research (or alternative) hypothesis. The null hypothesis is simply that the population means are equal&#8212;the observed difference is due to random error.&#8221;</p><p>And the one about the true value probably being within the confidence interval:</p><p>&#8220;A news story might report that a sample of people in your state found that 55% support a measure that will increase education funding in your state; 45% oppose the measure. The report then says that these results are accurate to within 3 percentage points, with a 95% confidence level. This means that the researchers are very (95%) confident that, if they were able to study the entire population rather than a sample, the actual percentage who support the education measure would be between 58% and 52% and the percentage opposing the measure would be between 48% and 42%.&#8221;</p><p>In a word, this book is absolute shit. Whatever lip service it gives the crisis and possible remedies with one hand, it undoes them with the other. The furthest it goes is to say that the result from the famous 2015 study of the replication rate in psychology was disappointing.</p><h2>The twist</h2><p>Of course we have to go where the data leads us and in this case the data almost made me fall off my chair. Paul Cozby wrote another &#8220;Methods in Behavioral Research&#8221; book that corrects all of these issues and more. It was written with Raymond Mar and it&#8217;s the Canadian edition of this book.</p><p>It&#8217;s not just good. It&#8217;s pixel perfect:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!3Skp!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!3Skp!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png 424w, https://substackcdn.com/image/fetch/$s_!3Skp!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png 848w, https://substackcdn.com/image/fetch/$s_!3Skp!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png 1272w, https://substackcdn.com/image/fetch/$s_!3Skp!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!3Skp!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png" width="657" height="745" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:745,&quot;width&quot;:657,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:339762,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/210097157?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!3Skp!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png 424w, https://substackcdn.com/image/fetch/$s_!3Skp!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png 848w, https://substackcdn.com/image/fetch/$s_!3Skp!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png 1272w, https://substackcdn.com/image/fetch/$s_!3Skp!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F97060340-cfb7-47eb-93a1-4f5a83ef59b8_657x745.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>&#8220;Research out of the University of Guelph also found that almost 90 percent of introductory psychology textbooks that define the p-value fail to define it correctly (Cassidy et al., 2019).&#8221; Indeed. Indeed.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Cd5p!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Cd5p!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg 424w, https://substackcdn.com/image/fetch/$s_!Cd5p!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg 848w, https://substackcdn.com/image/fetch/$s_!Cd5p!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!Cd5p!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!Cd5p!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg" width="720" height="942" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:942,&quot;width&quot;:720,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:736336,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/210097157?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Cd5p!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg 424w, https://substackcdn.com/image/fetch/$s_!Cd5p!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg 848w, https://substackcdn.com/image/fetch/$s_!Cd5p!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!Cd5p!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4f263cfc-6d18-4508-9901-4ab27653463e_720x942.jpeg 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption"><strong>Source:</strong> McGraw Hill, &#169; 2024 McGraw Hill.</figcaption></figure></div><p>So the only thing to do is to re-examine Alexander&#8217;s statement. It&#8217;s true that fields have confronted the replication crisis at different, wildly inappropriate and late, times. And they&#8217;ve adjusted in different ways at different times like the different ways that exist on one side of the Canadian border or the other, or one side of Paul Cozby&#8217;s desk or the other. The same ISBN is still being sold as 2026 &#8220;evergreen&#8221; by the publisher.</p><p>I don&#8217;t know how to put a finer point on this other than to say that the readers of this book will grow up to make false statements to our government for money. In Canada, maybe you&#8217;ll be okay. Just make sure you buy the right one.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.redteamofscience.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Subscribe to receive new posts and support my work. Educators, consider starting your own <a href="https://www.routledge.com/textbooks/evaluation/9781041218722">metascience course</a>.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div>]]></content:encoded></item><item><title><![CDATA[Public Health and the Right to Reason]]></title><description><![CDATA[Referenda on Tony Fauci]]></description><link>https://www.redteamofscience.com/p/public-health-and-the-right-to-reason</link><guid isPermaLink="false">https://www.redteamofscience.com/p/public-health-and-the-right-to-reason</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Wed, 05 Aug 2026 12:31:37 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!9P17!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>One of the most substantial events in modern science-of-science has concluded. Anthony Fauci&#8217;s testimony was both symbolic and literal, a referendum on trust in science. He is a symbol that scientists only care about knowledge; the stakes for themselves be damned. He was also the actual head of a large funder for 35 years and the most visible representative of science worldwide during the pandemic. The political stakes over funding, elitism, and expertise have been too tempting for us to leave alone.</p><p>Like most rational people, I put this diminutive old man in the middle of the pack of scientists at first. I believe that scientists care about knowledge and not stakes for themselves a little, and I guessed this was true for Fauci. Where we conflicted on COVID origin, I started to suspect a noble lie.</p><p>Like <a href="https://poll.qu.edu/poll-release?releaseid=3869">two-thirds</a> of Americans, I think the lab leak is more likely than a purely natural origin of the virus. <a href="https://www.science.org/content/article/virologists-and-epidemiologists-back-natural-origin-covid-19-survey-suggests">20% of experts agree</a>. It&#8217;s the more rational choice based on statistics we all know and use intuitively, exemplified by the fact that the most prominent user of this reasoning is a comedian.</p><p>This <a href="https://www.redteamofscience.com/p/covid-origin-may-be-simple">reasoning is simple</a> and nobody can say for sure whether COVID origin is simple, or not. Based on the communications that have now been released, scientists privately thought it was simple.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!9P17!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!9P17!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png 424w, https://substackcdn.com/image/fetch/$s_!9P17!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png 848w, https://substackcdn.com/image/fetch/$s_!9P17!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png 1272w, https://substackcdn.com/image/fetch/$s_!9P17!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!9P17!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png" width="1402" height="1122" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1122,&quot;width&quot;:1402,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:2571246,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/209781074?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!9P17!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png 424w, https://substackcdn.com/image/fetch/$s_!9P17!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png 848w, https://substackcdn.com/image/fetch/$s_!9P17!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png 1272w, https://substackcdn.com/image/fetch/$s_!9P17!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0af22830-91d4-4cd4-9500-9ec5ee6e3053_1402x1122.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Stewart as <a href="https://en.wikipedia.org/wiki/Sta%C5%84czyk_(painting)">Sta&#324;czyk</a> (Credit: ChatGPT)</figcaption></figure></div><h2>Naive probability</h2><p>Naive probability is a technical term for probability estimated with little information. Naive probability is counting things. On COVID, naive probability is easy to illustrate with dogs.</p><p>Rabies can be spread by dogs, but rabies is rare. You could reduce the chance of rabies by saying people can&#8217;t keep dogs as pets anymore. The Chinese government could have done this with wet markets, eliminating them to prevent a new SARS. But the Chinese people might disagree. &#8220;I go to my wet market all the time and I&#8217;ve never gotten sick.&#8221;</p><p>The Chinese people, and you if you love your family pet, are right. Each market is very safe on its own. Each dog is very safe. Which is why, if someone blamed your dog for a rabies outbreak and you happen to live next to a lab studying rabies, you have the right to complain. The outbreak most likely came from the rabies lab.</p><p>This is not nostalgia, you for your dog, and the Chinese for their market. It&#8217;s math. Each market is very, very safe. Each dog is very, very safe. So when weighing the probability of one dog versus one laboratory studying rabies, there&#8217;s no question that the dog is more safe. There are only a few labs per infection. Lab leaks are relatively common compared to dogs with rabies.</p><p>Consider planes. Planes are responsible for the worst imaginable fates and yet we still use them because they&#8217;re very safe. If a plane is flying over a dynamite factory and all you knew is that there was a loud boom, you would choose being on the plane over being in the factory knowing almost nothing about dynamite factories.</p><p>Some may disagree on the details, but we&#8217;re not talking about exact numbers. We&#8217;re talking about the war over using a mental tool like counting things at all, particularly in conjunction with another mental tool, conflict of interest.</p><p>It&#8217;s safe to say that all of the formal study of scientific practice is driven by an understanding of conflict of interest. Dr. Fauci wrote about it in his <a href="https://www.paul.senate.gov/wp-content/uploads/2026/07/2026.07.24_Tonys-Diary-Package.pdf">diary in February 2020</a>. He described a meeting of eleven top scientists in which only two thought deliberate insertion (gain of function) was off the table. &#8220;There was not total agreement about the likelihood of deliberate insertion. Ron Fouchier said he was sure that this could occur naturally and we should not waste our time and divert effort to pursue this.&#8221;</p><p>If you&#8217;re even moderately familiar with the field, you might be tempted to say, &#8220;Of course Ron Fouchier would say that,&#8221; and so did Fauci. Ron Fouchier was the first to insert DNA into a virus to enhance its ability to infect a human analog. &#8220;This is expected of him since he was the original GOF person with Yoshi Kawaoka.&#8221;</p><p>This may be the most ironic line in the diary because subsequently much of the world, roughly hundreds of millions of people, turned that reasoning on Fauci himself.</p><h2>Even worse coincidences</h2><p>Fauci is like us. He doesn&#8217;t want to have to trust experts absolutely given the high stakes involved for some of them. He also knows the statistics of coincidence and takes it even further: &#8220;...the rest felt that deliberate insertion was possible and given the fact that Dr. Zhen-Li Shi at the University of Wuhan has been working for years in GOF in coronaviruses to allow adaptation of the spike protein to bind to the human ACE2 receptor&#8230;&#8221; Nature could have come up with any number of known or unknown methods of infecting cells. COVID uses a particular one that scientists knew about, and the exact one that had been practiced at the nearby lab.</p><p>If the meeting attendees go down in history as heroes or villains, it should be for not being good mentors to the rest of us. If they&#8217;re going to use some mental tool, they should encourage us to use it too. Tony Fauci is not nearly the expert that Ron Fouchier is, but he doubted Fouchier&#8217;s expertise due to conflict of interest. He should let us do the same.</p><p>The scientists whose communications were released didn&#8217;t give up their right to conclude. Far from it. It seems to be universal that they thought virtually everyone is stupid but them, even though their conclusions were common. All of the normal conclusions were privately pondered by scientists. Fauci distrusted the Chinese, for instance. So much so that he tossed around the idea that the first COVID infection could have been six months earlier than the Chinese admitted based on the age of some mice that were engineered to contract similar disease and happened to be able to contract the new virus. Not dispositive proof, certainly. But it didn&#8217;t stop him from pondering.</p><div class="subscription-widget-wrap-editor" data-attrs="{&quot;url&quot;:&quot;https://www.redteamofscience.com/subscribe?&quot;,&quot;text&quot;:&quot;Subscribe&quot;,&quot;language&quot;:&quot;en&quot;}" data-component-name="SubscribeWidgetToDOM"><div class="subscription-widget show-subscribe"><div class="preamble"><p class="cta-caption">Support Red Team of Science by subscribing, or starting a <a href="https://www.routledge.com/textbooks/evaluation/9781041218722">metascience course</a>.</p></div><form class="subscription-widget-subscribe"><input type="email" class="email-input" name="email" placeholder="Type your email&#8230;" tabindex="-1"><input type="submit" class="button primary" value="Subscribe"><div class="fake-input-wrapper"><div class="fake-input"></div><div class="fake-button"></div></div></form></div></div><h2>The epicenter</h2><p>The danger in leaving simple reasoning to experts is vast. For one, we have to suffer through terrible papers. None more so than the one by University of Arizona&#8217;s Michael Worobey.</p><p>Many scientists agreed that the Chinese data was not trustworthy. They signed a <a href="https://pmc.ncbi.nlm.nih.gov/articles/PMC9520851/">petition</a> to that effect. Dr. Worobey, one of the co-authors, nonetheless snuck off and worked in his basement to produce a paper using the data he&#8217;d criticized. He claimed it could be used to pinpoint the epicenter of the outbreak, drawing concentric circles around the market as if the Chinese are little ants never going far from their food hole. We have to suppose the virus can travel thousands of miles in a day after the market, but never traveled 8 miles before it. It can travel 8 miles from a direction that&#8217;s not the lab, but it can&#8217;t travel 8 miles from the direction of the lab. To read this paper is to suffer as one of Worobey&#8217;s ants.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!_dVO!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!_dVO!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg 424w, https://substackcdn.com/image/fetch/$s_!_dVO!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg 848w, https://substackcdn.com/image/fetch/$s_!_dVO!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!_dVO!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!_dVO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg" width="1456" height="1092" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1092,&quot;width&quot;:1456,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:4160042,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/209781074?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!_dVO!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg 424w, https://substackcdn.com/image/fetch/$s_!_dVO!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg 848w, https://substackcdn.com/image/fetch/$s_!_dVO!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!_dVO!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe28e4916-4735-442f-b8dc-f8b17f00e7b2_3648x2736.jpeg 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">The Wuhan skyline</figcaption></figure></div><p>It&#8217;s safe to say that Tony Fauci is familiar with the idea that scientists and governments may lie, or weigh some high stakes for themselves above the truth, or innocently kid themselves. It&#8217;s safe to say that he has faced the temptation of a noble lie. Like many public health officials, he probably weighs the tradeoffs differently from the rest of us. But there&#8217;s a spectrum of noble lies. On one end we say we&#8217;re having a great day when we aren&#8217;t. On the other, we become strangers to our own brains. That we should leave simple reasoning to scientists is on the bad end.</p><p>So last week Tony Fauci sipped lukewarm water in front of the cameras no longer needed as a truth teller, or a liar. What he represents now is the character of science in the face of high stakes. How much can we trust someone when the stakes are so high? What if they convince themselves of something untrue? Or morally right?</p><p>Dr. Fauci didn&#8217;t speak at the hearings but he will no doubt speak to us again and the stakes are very high, maybe too high for one person. What he represents is no less than the warranty of the human brain. The worst outcome may be false hope that it was all true.</p>]]></content:encoded></item><item><title><![CDATA[The University as Territory]]></title><description><![CDATA[The Vanderbilt Report is another attempt to storm the gates in a war over nonsense]]></description><link>https://www.redteamofscience.com/p/the-university-as-territory</link><guid isPermaLink="false">https://www.redteamofscience.com/p/the-university-as-territory</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Wed, 22 Jul 2026 13:03:06 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!Hr48!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The Vanderbilt Report is a 30-page document commissioned by Vanderbilt University. It&#8217;s been called &#8220;<a href="https://www.chronicle.com/article/are-leftist-professors-really-destroying-the-academys-credibility">politely worded McCarthyism</a>,&#8221; &#8220;<a href="https://www.insidehighered.com/news/faculty/research/2026/06/10/professors-say-vanderbilt-report-misrepresents-their-work">unhinged</a>,&#8221; and &#8220;<a href="https://www.chronicle.com/article/inside-the-report-thats-divided-the-academy">diabolically evil</a>.&#8221; It prompted nine articles and op-eds at the Chronicle alone over the last six weeks.</p><p>On the other side is the insurgency. Like-minded organization Heterodox Academy praised the report, and hosted perhaps its only <a href="https://heterodoxacademy.org/events/the-humanities-in-crisis-a-discussion-on-the-state-of-scholarship-in-the-humanities-report/">debate</a>. At the debate and in much of the work that inspired it, ideas like evolution and intersubjective agreement are used as little as possible as if too hot to touch.</p><p>Thirty years ago, this set of facts would have been hard to follow. It would have been hard to imagine what&#8217;s in the report, what&#8217;s considered heterodox these days, and why anyone would be mad. Today, however, those familiar with higher education can guess what&#8217;s in the report, who is calling it evil and why.</p><p>Those even more familiar with higher education might be able to guess the unintuitive reality that <em>both sides</em> treat biology, evolution, and intersubjective agreement (replication, post-publication review and the like) as unclean, something others might do. Empiricism, or the learning-from-observation claimed in funding proposals, is almost never mentioned. On the methods used in the Vanderbilt Report, co-author Ashley Rubin said that one method would have been to &#8220;read studies and evaluate them based on scholarly standards.&#8221; But &#8220;people look at this work with great skepticism. There&#8217;s a big cost to doing that type of work.&#8221; Rubin also notes that evolution and biology are seen as &#8220;conservative and therefore bad.&#8221;</p><p>Maybe Rubin said it too strongly, but it&#8217;s true that previous work, the Vanderbilt Report, insurgency groups like Heterodox Academy, and the beliefs they generally oppose like relativism, postmodernism, constructivism all don&#8217;t want to be too closely associated with the philosophy that built the Enlightenment and the idea of the university in the first place.</p><p>After empiricism built the universities, they became a thing one can take and hold and not let others in. We know this because they were taken over by nonsense and now nonsense doesn&#8217;t want to let go. Universities are territory.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Hr48!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Hr48!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg 424w, https://substackcdn.com/image/fetch/$s_!Hr48!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg 848w, https://substackcdn.com/image/fetch/$s_!Hr48!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!Hr48!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Hr48!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg" width="640" height="479" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:479,&quot;width&quot;:640,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:110046,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/207824664?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Hr48!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg 424w, https://substackcdn.com/image/fetch/$s_!Hr48!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg 848w, https://substackcdn.com/image/fetch/$s_!Hr48!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!Hr48!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa953c7f7-af0d-49c0-920e-c7aba89805f4_640x479.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Andrew Telfer / <em>Trinity College, Oxford</em> / CC BY-SA 2.0</figcaption></figure></div><h2>The insurgent ideas</h2><p>According to the Vanderbilt Report&#8217;s lead author, the philosophies that have taken over universities, relativism and postmodernism, have almost no value whatsoever. Paul Boghossian&#8217;s 2006 book <em>Fear of Knowledge</em> shows that they are contradictory and can&#8217;t be accepted under any system of belief that includes <em>contradiction</em>, or simply <em>reality separate from ourselves</em>. The eminent NYU philosopher has therefore spent many years of his life becoming an expert on nothing.</p><p>The book is excellent, but its arguments are focused on contradiction, which doesn&#8217;t seem to bother his opponents. It waits until the epilogue to say what we&#8217;re all thinking, that these ideas have &#8220;come to tempt so many&#8221; partially because &#8220;they are hugely empowering.&#8221;</p><p>You could say the book is focused on being right instead of being strong. For example, reading a history of the early Scientific Revolution into the start of the Industrial Revolution, one might say, &#8220;Well. The Catholic Church is toast.&#8221; One would of course have been wrong, but there&#8217;s a strong argument that the church would fade away: Putting nature in a controlled state also means that one can put a useful object like a steam engine in a controlled state too and predict that it will trundle down the tracks. There&#8217;s something essential to human life that would need to be subverted for humans to <em>not</em> use science to build steam engines and so they&#8217;re going to and therefore the early opponent of the scientific revolution, the Catholic Church, is toast.</p><p>You would have been wrong, but it&#8217;s not a bad guess. Boghossian makes an argument about the similarities between postmodernism and religion too and it&#8217;s truly beautiful:</p><p>&#8220;Yes, the Cardinal consults his Bible to find out what to believe about the heavens, rather than using the telescope; but he doesn&#8217;t divine what the Bible itself contains, but rather reads it using his eyes. Nor does he check it every hour to make sure that it still says the same, but rather relies on induction to predict that it will say the same tomorrow as it does today.&#8221;</p><p>Where Boghossian lacks imagination, though, is in enumerating the set of belief systems from which relativism is drawn. This is important because it connects to the reason why scientific thought lost out to postmodernism and relativism in universities, and why the territory may be lost again, even if Boghossian and others like him win.</p><p>Postmodernism is Scientology. It&#8217;s Catholic. It&#8217;s countless fundamentalist churches. It is, as the Vanderbilt Report says, astrology. Non-empiricism is all of a piece, but it is not the same. Postmodernism is therefore chosen from an infinite number of belief systems that can&#8217;t be proven wrong. That&#8217;s all well and good. We knew that. But what are the chances that postmodernism wasn&#8217;t therefore <em>designed</em> to take over universities? It&#8217;s a belief system that only exists at universities.</p><p><em>Designed</em> doesn&#8217;t need to be <em>purposefully designed</em>. It could have evolved at universities. But either way, it amounts to the same thing because true philosophies can&#8217;t be for a purpose unless a radical coincidence has occurred. What are the chances that a belief system perfectly serves the personal interests of a specific group of people measured in money and status, and also happens to be true?</p><p>To say that postmodernism is intentional is a strong argument. It&#8217;s not that sophisticated and yet these kinds of arguments are generally avoided by the insurgency as if rude.</p><h2>Postmodern relativism</h2><p>We have seen how a system like postmodernism works. Intellectuals who win debates deserve our respect; Postmodernists win debates by accusations severe enough to shut the debate down. It&#8217;s physical action, having the opponents hauled away, disguised as argument. Similarly, relativists choose some group of people for empiricists to squabble with while they assume the throne. Throughout <em>Fear of Knowledge,</em> the group relativists have sent to fight empiricists is the Azande tribe, whom Boghossian ably and politely refutes.</p><p>The system he&#8217;s responding to is well known at this point. In order to argue against a relativist or postmodernist, one needs to dance with the devil of oppression. Not unlike the preachers of old and the literal devil, postmodernists know they can accuse their opponents of unspeakable horrors. Readers of postmodernism probably detected how strong this tactic is in the 1970s and thought, &#8220;Well. Objectivity is toast.&#8221; It wouldn&#8217;t have been a bad guess.</p><h2>An alternative to Vanderbilt: make predictions</h2><p>What Boghossian missed out on is borrowing from the empiricist tradition to which he belongs and making a prediction. Here&#8217;s a suggestion: &#8220;Now I know that what will likely happen is that anti-empiricists get to wait until this report makes a big splash, see if it&#8217;s bad for themselves personally, and react with the shortest path between rhetoric and action to claim that I, Boghossian, coincidentally am an immediate danger to life and limb and must be expunged from society. This is, in a modern civilized world, physical action disguised as argument.&#8221;</p><p>Why make a prediction? It&#8217;s easy to say now that this prediction came almost perfectly true, with the head of the American Anthropological Association <a href="https://www.chronicle.com/article/is-anthropology-hopelessly-politicized">implying</a> the opposition wants to debate Nazism, internment, and potentially kill babies who fit outside the sex binary.</p><p>Regardless, we knew already on principle that prediction was the way to go. Instead of prediction, what we got was Boghossian and co-author Kwame Anthony Appiah saying that departments should begin &#8220;internal dialogues,&#8221; and co-author Ashley Rubin&#8217;s phrasing, &#8220;faculty-led self-study.&#8221; These will fail for obvious reasons.</p><p>This is not to say that Boghossian is unaware of the real world, nor is it to say that the insurgency should use the same tactics as postmodernists. But ignoring the real physical world blocks off a whole universe of generalization about anti-empiricism. For one, the empiricist can make a fairly useful generalization about anti-empiricists that for any living thing to survive, it needs some way to predict what&#8217;s going to happen to find food. If an organism like postmodernism has survived for this long without prediction, it&#8217;s going to need some alternative source of food. Its food, unfortunately, is Boghossian.</p><p>Again, this is not to say that Boghossian and co-authors don&#8217;t understand this. They understand it a little too well. They probably know that if they come out swinging, they&#8217;re going to be eaten, and if they don&#8217;t &#8212; they don&#8217;t go quite to the front lines &#8212; then someone is going to do their fighting for them. We have to protect our society after all. We have empirical courts and medicine and basketball games to protect. Someone has to push back.</p><p>That&#8217;s all well and good. Let&#8217;s be philosophers and not strategists. But there&#8217;s another uncomfortable truth, which is that we have to recognize the coincidence that if any strategy were to succeed in getting anti-empiricists out of academia, these specific empiricists would be first in line to benefit.</p><p>Heterodox Academy, for instance, is a member-supported organization of people who work at universities who would be first in line. In other words, the same people who folded to postmodernists in the first place. The same people who, in many cases, are the few people on Earth who don&#8217;t need to worry about being fired. They are going to benefit from the difficult truths that only the rest of us really need to face or, god forbid, write down.</p><p>Ordinary people who don&#8217;t have to worry about offending a relativist are more willing to draw the line at evolution and intersubjective agreement because they have to. They have to find their own food and survive. The fact that these basic scientific pillars would be considered too heterodox for a Heterodox Academy member or associated with conservatism is &#8230; unexpected. Thirty years ago, saying that conservatism and evolution go together would be laughable.</p><h2>The university as territory</h2><p>So what drives the insurgency if not empiricism? The insurgency is driven by territory, almost by default. This again is given by evolution. That which survives in academia wins.</p><p>If you don&#8217;t already have a little territory at a university, you need not apply to Heterodox Academy. Like postmodernists, Heterodox Academy seems keenly aware of the idea of territory if you read between the lines. Their principles are designed to keep the territory intact: correct, but bland. Open inquiry, viewpoint diversity. These two phrases have already been banned, but they&#8217;re hard to disagree with. In fact, if postmodernists were powerless and rare in academia, Heterodox Academy could be repurposed for their interests.</p><p>That&#8217;s the problem with not having something that you stand for. Without a definition, the insurgency may simply evolve into that which survives best in academia. With the Vanderbilt Report, the insurgency is becoming a land of undercurrents, of hints and intimation, whipping through techniques that didn&#8217;t win. Replication is so yesterday.</p><p>Heterodox Academy president John Tomasi seems to be testing out a novel tactic, saying, &#8220;Reading the report&#8217;s opening pages, I imagine myself in an old-time Western, out by a fire on the prairie, with a philosopher-sheriff assembling a posse to round up a band of disciplinary desperados and bring them to scholarly justice.&#8221;</p><p>We can&#8217;t read too far into what he means, but let&#8217;s get this out of the way. Boghossian is no sheriff, and as soon as Boghossian fails, another hero will be chosen on and on until the Chronicle runs out of hyperbole to print about them.</p><p>This will only end one way, with another philosophy emerging that is designed to take over.</p><p>The conclusion has been foregone from the beginning that there&#8217;s no way to defeat nonsense. The Catholic Church didn&#8217;t give up and neither will postmodernists. The way the church was limited was by building something that&#8217;s not a church, a place where appealing to god is just not done.</p><p>Unless universities are willing to say they&#8217;re empirical the way the church says it&#8217;s holy, or postmodernists say they&#8217;re righteous, we&#8217;re going to have to build something that&#8217;s not a university. This is not a revolution. It&#8217;s simply going to happen. Empiricism will move elsewhere, if it hasn&#8217;t already.</p><p></p>]]></content:encoded></item><item><title><![CDATA[The missing link]]></title><description><![CDATA[A poll finds that priming research might be bad enough to save science]]></description><link>https://www.redteamofscience.com/p/the-missing-link</link><guid isPermaLink="false">https://www.redteamofscience.com/p/the-missing-link</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Mon, 16 Mar 2026 13:20:23 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!o9Nn!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Priming in psychology is one of the least respected areas of research. Psychologists have a very dim view of priming, and yet priming research continues. A recent <a href="https://replications.clearerthinking.org/how-much-do-academic-psychologists-trust-academic-psychology-and-is-there-still-a-replication-crisis/#anonymized--csv-dataset">poll</a> asked psychologists if priming is real and meaningful and only 5% said yes. 5% is in line with belief in <a href="https://psi-encyclopedia.spr.ac.uk/articles/parapsychology-psychology-textbooks/">ESP</a> among researchers.</p><p>Priming is subtle suggestions in a paragraph, or a room, or a question that cause nebulous and just as subtle changes in behavior in someone who experiences them.</p><p>Why does this research persist despite it being so unlikely to be real? The answer, like many things, is p-values. Luckily, this poll asks approximately the right question so that we can translate the responses into math and say what p-values mean in terms of what the public understands: whether or not something is true.</p><p>To do this, we have to fight through a problem known since the 1920s, that p-values don&#8217;t guarantee anything, and even if we use what we&#8217;re really talking about, which is effect size, the sizes don&#8217;t correspond to &#8220;care&#8221; or &#8220;don&#8217;t care.&#8221; For example, psychology is undecided as to whether we should care about an effect of 0.1, whether it is important, or random. Gone are the days of effects big enough for grand public demonstration, of dino bones and electromagnetic fields. We don&#8217;t have a game show called RESIST. THAT. PRIME!</p><p>We can&#8217;t demonstrate anything with effect sizes, but we can use two numbers that are unambiguous: the ordinary threshold for publication (p &lt; 0.05), and the acceptable proportion of false findings in the literature, and say that priming research is beyond repair.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!o9Nn!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!o9Nn!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg 424w, https://substackcdn.com/image/fetch/$s_!o9Nn!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg 848w, https://substackcdn.com/image/fetch/$s_!o9Nn!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!o9Nn!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!o9Nn!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg" width="1248" height="832" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/df473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:832,&quot;width&quot;:1248,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:227889,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/190737369?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!o9Nn!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg 424w, https://substackcdn.com/image/fetch/$s_!o9Nn!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg 848w, https://substackcdn.com/image/fetch/$s_!o9Nn!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!o9Nn!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fdf473448-8f8f-467e-b04d-4a397aa0c8d8_1248x832.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The proportion of false findings for all of published science was estimated to be <a href="https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124">50%</a>. It&#8217;s a number very difficult to estimate science-wide, but we can agree that 50% false positives is bad. The ordinary interpretation of publication thresholds is that false positives occur only about 5% of the time.</p><p>For the portion of published findings that are false, let&#8217;s assume this interpretation:</p><div class="pullquote"><p>5% is reasonable. 50% is terrible.</p></div><p>In a single field with a single distribution of how big the effect probably is, you can control the rate of false discoveries by raising the standards for publication. So how high would the standards have to be in priming to keep the false discovery rate at a reasonable 5% level?</p><p>Here&#8217;s the distribution implied by the poll, counting only researchers who had an opinion:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!g3t3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!g3t3!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png 424w, https://substackcdn.com/image/fetch/$s_!g3t3!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png 848w, https://substackcdn.com/image/fetch/$s_!g3t3!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png 1272w, https://substackcdn.com/image/fetch/$s_!g3t3!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!g3t3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png" width="835" height="822" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:822,&quot;width&quot;:835,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:14143,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/190737369?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!g3t3!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png 424w, https://substackcdn.com/image/fetch/$s_!g3t3!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png 848w, https://substackcdn.com/image/fetch/$s_!g3t3!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png 1272w, https://substackcdn.com/image/fetch/$s_!g3t3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6198ebcf-d76e-417c-921a-47c690ea586a_835x822.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>By these estimates, no feasible standard would keep priming research at a reasonable level of false discovery. p-values would have to be lower than 0.001, 50 times stricter than today&#8217;s standards.</p><p>On the other hand, if you&#8217;re fine with 50% of research being false, then anything goes. Even a field as unlikely to be real and meaningful as priming can go on at current standards for publication, or even weaker standards. This is why accepting that half of research is not true is unreasonable, because almost no avenue of inquiry will ever be abandoned.</p><p>Worse, if you keep accepting the same standard of evidence, your ESP papers and your priming papers become surprising and &#8220;important&#8221; because ESP and priming are unlikely to be real.</p><p></p><h2>The power of post hoc</h2><p>This analysis depends on assumptions like any other and the poll is small. If we want more precise numbers, we should ask for exact estimates instead of buckets like &#8220;tiny.&#8221; And it assumes sample sizes stay around what they are in psychology. (If you keep p-value standards the same but increase the sample size standard instead, priming studies would cost about 6 times more than they do now.)</p><p>Additionally, there are so many researchers who are on the fence in the poll that interpreting &#8220;tiny&#8221; as 0.02 yields different results compared to 0.1. This is important because effect sizes of 0.1 are called &#8220;<a href="https://doi.org/10.1177/1745691620984483">indispensable</a>&#8221; or &#8220;<a href="https://psycnet.apa.org/doi/10.1037/pro0000386">crud</a>&#8221; depending on who you ask. Despite anchoring to indisputable numbers, we&#8217;re back to arguing what &#8220;important&#8221; means. If respondents on the fence meant tiny is an effect size of 0.1 and the top group meant 0.2, then anything goes and current standards are fine, even for priming.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!VkVj!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!VkVj!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png 424w, https://substackcdn.com/image/fetch/$s_!VkVj!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png 848w, https://substackcdn.com/image/fetch/$s_!VkVj!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png 1272w, https://substackcdn.com/image/fetch/$s_!VkVj!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!VkVj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png" width="837" height="837" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:837,&quot;width&quot;:837,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:47106,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/190737369?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!VkVj!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png 424w, https://substackcdn.com/image/fetch/$s_!VkVj!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png 848w, https://substackcdn.com/image/fetch/$s_!VkVj!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png 1272w, https://substackcdn.com/image/fetch/$s_!VkVj!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe8aa74c3-5eb3-4d17-8461-d30a58f5cc57_837x837.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Note: Only the &#8220;25% false&#8221; chart is interesting. 5% is almost all red, and 50% is all a &#8220;green light&#8221; for more priming research.</figcaption></figure></div><p>The consequences of being fine with 50% false research should seem eerily similar to the world we live in where anything you&#8217;d like to be true can be found in the literature somewhere. It&#8217;s also interesting to note that the entire range of implied standards, from open science radical to status quo radical, depends on whether you call 0.1 or 0.02 &#8220;tiny.&#8221;</p><p>So we must all live in awe of the immense power of post hoc reasoning that can change the definition of tiny for the purpose of publication. After the fact, one can simply say 0.1 is important. Findings that many would argue are indistinguishable from noise are suddenly essential building blocks that count in priming&#8217;s favor. Journals and researchers are equally innocent on this point since each can blame the other. The only anchor in the epistemic stew is that priming is really one of the worst. There was a replication crisis over it.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!t8pN!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!t8pN!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png 424w, https://substackcdn.com/image/fetch/$s_!t8pN!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png 848w, https://substackcdn.com/image/fetch/$s_!t8pN!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png 1272w, https://substackcdn.com/image/fetch/$s_!t8pN!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!t8pN!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png" width="885" height="507" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/bccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:507,&quot;width&quot;:885,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:32587,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/190737369?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!t8pN!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png 424w, https://substackcdn.com/image/fetch/$s_!t8pN!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png 848w, https://substackcdn.com/image/fetch/$s_!t8pN!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png 1272w, https://substackcdn.com/image/fetch/$s_!t8pN!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fbccfffa4-fbd6-46e3-966f-5eda4c54d76b_885x507.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Papers mentioning &#8220;social priming&#8221; per year. Source: Scopus</figcaption></figure></div><p></p><h2>Rebuttal</h2><p>A common rebuttal here is to say that researchers weren&#8217;t interpreting the 5% p-value standard as a 5% chance the finding is false. p-value standards don&#8217;t guarantee anything, actually. In fact, one could say at this late date that they have always assumed that half of published research is false and p-values are just a convention to weed out completely ridiculous claims. It would be very late as p-values just turned 100.</p><p>A journal caught up in a <a href="https://www.newyorker.com/magazine/2026/02/02/did-a-celebrated-researcher-obscure-a-fatal-poisoning">scandal</a> covered by The New Yorker made this move a few weeks ago. They declared that it wasn&#8217;t that its case report was not true. <em>All</em> of its case reports <a href="https://www.the-scientist.com/138-case-reports-in-a-pediatric-medical-journal-turned-out-to-be-fake-74170">aren&#8217;t true</a>. They just neglected to say so. Again, we stand in awe of the power of post hoc reasoning. Not saying so in advance truly runs the world.</p><p>According to the poll, there really isn&#8217;t a compromise among the public, journals, and priming research. One has to go. You can&#8217;t tell the public that 50% of what they pay for is false and standards don&#8217;t need to go up as a field becomes more implausible. You can&#8217;t get journals to agree to 10% because it would mean publishing less and admitting 10% is actually quite strict. Even an optimistic estimate put science-wide falseness at <a href="https://doi.org/10.1093/biostatistics/kxt007">14%</a>. If you could get the public and journals to agree to something like 10% that is a real stretch for both parties, priming would still be wiped out.</p><p>The larger point is that we&#8217;re not sure the public would approve of p-values at all. Not without an estimate of what they mean about a field. We also don&#8217;t know what reform would make this estimating work, and who does it, but the information we need is in the poll.</p><p></p>]]></content:encoded></item><item><title><![CDATA[It always ends with numbers]]></title><description><![CDATA[How NIH&#8217;s project to save neuroscience education is switching sides]]></description><link>https://www.redteamofscience.com/p/it-always-ends-with-numbers</link><guid isPermaLink="false">https://www.redteamofscience.com/p/it-always-ends-with-numbers</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Wed, 04 Mar 2026 13:03:26 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!XeOY!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Science reform is hard. Even the largest funder in the world struggles with it. But one of the most expensive projects on science reform, NIH&#8217;s Community for Rigor has been a success in one way. It showed that when you try to take on what it called &#8220;the average scientist,&#8221; the average scientist fights back. Average scientists work their way into your project. Average scientists don&#8217;t like feedback. Average scientists really don&#8217;t like you insulting science. What the $16 million project has shown is something we already knew: questionable research is too common. It&#8217;s too average. Each time you look at Community for Rigor, the project is showing in a new way that it&#8217;s easier to join the mob than to push back.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!XeOY!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!XeOY!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg 424w, https://substackcdn.com/image/fetch/$s_!XeOY!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg 848w, https://substackcdn.com/image/fetch/$s_!XeOY!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!XeOY!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!XeOY!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg" width="1248" height="832" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/f36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:832,&quot;width&quot;:1248,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:181471,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/189571721?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!XeOY!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg 424w, https://substackcdn.com/image/fetch/$s_!XeOY!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg 848w, https://substackcdn.com/image/fetch/$s_!XeOY!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!XeOY!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff36860c2-e25f-440a-85cd-b39ab33d6045_1248x832.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">The Average Monster coming to kill your reform idea.</figcaption></figure></div><p>In another stunning act of normalcy, the project doesn&#8217;t seem to want its emails released. The grantees claimed theirs were proprietary. (NIH can&#8217;t do this for its own emails.) The shocking part is how you can really dive in anywhere into science and you eventually get to the same place: fudging the numbers. From my long running FOIA request:</p><div><hr></div><p>February 25, 2026</p><p>Valery,<br><br>Have you had any luck with your investigation? Normally I would send you a blog post for comment. This time the email is the blog post. Please let me know if there&#8217;s anything you don&#8217;t want published in your reply.<br><br>To recap, I was told my FOIA request for emails about NIH&#8217;s Community for Rigor project was returning too many records to be processed. This is because the short name for it, &#8220;C4R&#8221; was matching what the FOIA office called &#8220;junk.&#8221; About a month later, you sent the number of records for various searches. The number of records for the term &#8220;C4R&#8221; in conjunction with other search terms was lower than &#8220;C4R&#8221; alone. I told you this was impossible by normal search term logic and you said you had noticed this already and were investigating. That was two weeks ago.<br><br>The format of the numbers was different for one of the search results as if someone had copied some numbers directly and typed one by hand. I think most people would conclude that the likely explanation is that someone wrote down the wrong information.</p><div class="captioned-image-container"><figure><div class="image-link image2" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!9Zos!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png 424w, https://substackcdn.com/image/fetch/$s_!9Zos!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png 848w, https://substackcdn.com/image/fetch/$s_!9Zos!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png 1272w, https://substackcdn.com/image/fetch/$s_!9Zos!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!9Zos!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png" width="721" height="214" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/7a486fda-d101-4333-8624-baad3bb6090a_721x214.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:214,&quot;width&quot;:721,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:26644,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/189571721?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!9Zos!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png 424w, https://substackcdn.com/image/fetch/$s_!9Zos!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png 848w, https://substackcdn.com/image/fetch/$s_!9Zos!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png 1272w, https://substackcdn.com/image/fetch/$s_!9Zos!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a486fda-d101-4333-8624-baad3bb6090a_721x214.png 1456w" sizes="100vw"></picture><div></div></div></div></figure></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!h3zW!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!h3zW!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png 424w, https://substackcdn.com/image/fetch/$s_!h3zW!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png 848w, https://substackcdn.com/image/fetch/$s_!h3zW!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png 1272w, https://substackcdn.com/image/fetch/$s_!h3zW!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!h3zW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png" width="721" height="587" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:587,&quot;width&quot;:721,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:71929,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/189571721?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!h3zW!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png 424w, https://substackcdn.com/image/fetch/$s_!h3zW!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png 848w, https://substackcdn.com/image/fetch/$s_!h3zW!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png 1272w, https://substackcdn.com/image/fetch/$s_!h3zW!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F46f95c20-9c60-46f0-9a1e-cb63d4d3fc03_721x587.png 1456w" sizes="100vw"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">[Red circle will be important later.]</figcaption></figure></div><p>To further recap, Community for Rigor is one of the largest metascience projects ever funded at around 16 million dollars. The goal was to produce training material on rigorous scientific practices. It was designed so that NIH would be co-managing the project with the principal investigator at the University of Pennsylvania.<br><br>To most people who have had direct contact with science reform, it should have been clear that the project would be a failure. I think that NIH, and the eventual head of the project, Konrad Kording were finding out in real time how big the scandal in science is. Current NIH Director Jay Bhattacharya said recently that researchers might not find doing the rigorous practice of replication glamorous and may avoid this kind of project, even if it&#8217;s funded. All of these people were no stranger to problems in science, but maybe they didn&#8217;t know how ubiquitous and difficult these problems are.<br><br>The funding director Walter Koroshetz once implied some NIH grantees are &#8220;<a href="https://videocast.nih.gov/watch=45245">crooks</a>.&#8221; Kording wrote about how researchers can&#8217;t understand the brain because they can&#8217;t even understand a <a href="https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005268">microprocessor</a>. One of the grantees under this project is famous for writing a book called &#8220;<a href="https://www.callingbullshit.org/">Calling Bullshit</a>.&#8221; And Bhattacharya has had almost the entire scientific community denounce him. Maybe these people were aware there are problems, but they&#8217;ve never had to get researchers to actually follow rigorous practices that might be harmful to the researcher&#8217;s career. It&#8217;s easy to talk about abstractly. Getting a particular person to follow through and undermine their own reputation, or their employer&#8217;s, is very difficult.<br><br>The purpose is to say, as I&#8217;ve said before, you are not the story. I don&#8217;t really care if even the worst interpretation of this email chain is true, that the FOIA office is falsifying records. As you probably know, it would cost me thousands of dollars just to threaten to sue. And on the scale of NIH FOIA scandals, this one is tiny. Two presidents and two congresses have tried to get NIH to release COVID origin records and often failed. (I can only speculate on what&#8217;s in them, but for some reason, NIH has not been cooperative.) I spoke with the person who would know best on this issue, and he told me I&#8217;m getting &#8220;the runaround&#8221; from NIH.<br><br>What I do care about is what NIH has said it cares about, that the average scientist doesn&#8217;t know how to apply the scientific method. This wording, &#8220;the average scientist,&#8221; is from the <a href="https://reporter.nih.gov/search/8GcwvyqmYkaZeHzB9AehJA/project-details/10673711#description">C4R funding proposal</a>. What NIH, Konrad Kording, Walter Koroshetz, maybe even Jay Bhattacharya found out is that <em>the best scientists</em> handpicked by NIH don&#8217;t know how to apply the scientific method, or don&#8217;t want to.<br><br>Case in point, the slide from the Community for Rigor conference that started this is categorically wrong:<br></p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!0Df5!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!0Df5!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png 424w, https://substackcdn.com/image/fetch/$s_!0Df5!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png 848w, https://substackcdn.com/image/fetch/$s_!0Df5!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png 1272w, https://substackcdn.com/image/fetch/$s_!0Df5!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!0Df5!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png" width="1164" height="663" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:663,&quot;width&quot;:1164,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:358875,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/189571721?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!0Df5!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png 424w, https://substackcdn.com/image/fetch/$s_!0Df5!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png 848w, https://substackcdn.com/image/fetch/$s_!0Df5!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png 1272w, https://substackcdn.com/image/fetch/$s_!0Df5!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9780a340-8697-4709-9c82-046e3074d2fe_1164x663.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p><br>The image doesn&#8217;t depict rigorous practices and project leaders at NIH know this. When the NIH project leaders give talks, they present this version of the scientific lifecycle:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!0I6Y!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!0I6Y!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png 424w, https://substackcdn.com/image/fetch/$s_!0I6Y!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png 848w, https://substackcdn.com/image/fetch/$s_!0I6Y!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png 1272w, https://substackcdn.com/image/fetch/$s_!0I6Y!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!0I6Y!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png" width="1355" height="809" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:809,&quot;width&quot;:1355,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:499318,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/189571721?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!0I6Y!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png 424w, https://substackcdn.com/image/fetch/$s_!0I6Y!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png 848w, https://substackcdn.com/image/fetch/$s_!0I6Y!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png 1272w, https://substackcdn.com/image/fetch/$s_!0I6Y!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2b5ec29a-d3e5-492f-8a54-c61dc5f1f23a_1355x809.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>What the image above depicts is the head of a lab doing what has always worked: have a stream of grad students and postdocs attempt to farm data sets for publishable results. First one wins. Don&#8217;t worry what stage of the process you&#8217;re in. They&#8217;re all &#8220;on ramps&#8221; and you can always &#8220;revise your hypothesis.&#8221; The slide is what has, and perhaps should, end your career. It ended the career of USDA leader Brian Wansink who was just doing what everyone else was doing. It shouldn&#8217;t be part of the handpicked project to educate researchers on rigorous practices.<br><br>After I emailed Community for Rigor about this and similar issues throughout the conference videos, the videos were removed from youtube without explanation.<br><br>We&#8217;ve known about how common poor practices are for a while. The reason why the scandal didn&#8217;t resolve itself is because everyone is doing it. There weren&#8217;t bad apples to weed out because the bad apple view of science is wrong.<br><br>I hope you and my readers can tolerate a little grandstanding here. My point is that I don&#8217;t want to get involved in every nook and cranny of this story, and that is where this conversation is headed. I can&#8217;t figure out if the discrepancy is due to you, or Lauren, or the central search department. Common sense says that you&#8217;re just the latest people who don&#8217;t want damage to your reputation or to science&#8217;s reputation and you&#8217;re not going to return the records for some reason. All I can do is keep asking and underscoring how important it is that you follow through.<br><br>However, the longer this goes on, the clearer it will be that the problem is the average scientist just as NIH believes by funding this project. This could go on for another year, and really it&#8217;s money well spent because NIH did an experiment by funding this project and these are the results. Hidden videos, <a href="https://web.archive.org/web/20250912000608/https://www.c4r.io/units">hidden materials</a>, and of course emails.<br><br>Please let me know about the results of your investigation and whether or not you have an estimate for completing the request. In case it wasn&#8217;t clear, I am not blaming you or Lauren for the delays. I have great respect for all of the people I listed above and I wouldn&#8217;t blame them either. You&#8217;re welcome to ask to be anonymous, and if you want to hand me off to someone else, I&#8217;m happy to continue with them.<br></p><p>Best,<br><br>Alex</p><div><hr></div><p></p><h2>The numbers</h2><p>The NIH FOIA office usually changes the subject without commenting on previous implausibilities and they did so in this case. They are most certainly writing down the wrong information, though. A few days later, I got a reply that included statistics for the same searches again:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!oThL!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!oThL!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png 424w, https://substackcdn.com/image/fetch/$s_!oThL!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png 848w, https://substackcdn.com/image/fetch/$s_!oThL!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png 1272w, https://substackcdn.com/image/fetch/$s_!oThL!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!oThL!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png" width="524" height="611" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:611,&quot;width&quot;:524,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:67202,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/189571721?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!oThL!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png 424w, https://substackcdn.com/image/fetch/$s_!oThL!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png 848w, https://substackcdn.com/image/fetch/$s_!oThL!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png 1272w, https://substackcdn.com/image/fetch/$s_!oThL!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F4930a2b1-b83d-4597-b036-a2eca9f04579_524x611.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!9IYV!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!9IYV!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png 424w, https://substackcdn.com/image/fetch/$s_!9IYV!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png 848w, https://substackcdn.com/image/fetch/$s_!9IYV!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png 1272w, https://substackcdn.com/image/fetch/$s_!9IYV!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!9IYV!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png" width="524" height="660" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:660,&quot;width&quot;:524,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:70697,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/189571721?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!9IYV!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png 424w, https://substackcdn.com/image/fetch/$s_!9IYV!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png 848w, https://substackcdn.com/image/fetch/$s_!9IYV!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png 1272w, https://substackcdn.com/image/fetch/$s_!9IYV!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F68563987-a9fa-4e00-b484-33e13a4af6e7_524x660.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The numbers had been changed and the formatting was consistent this time. There was no mention of the changes in the email. The impossibility of adding more &#8220;OR&#8221; terms and getting fewer results had been fixed, so to speak, due to the deletion of 517 emails from program officer Devon Crawford&#8217;s total.</p><p>Did someone delete 517 emails from Dr. Crawford&#8217;s mailbox? It seems unlikely since the other search should have been reduced by 517 too. After a year of this FOIA request and eight months of writing about it, those emails disappeared just as I complained that the numbers don&#8217;t make logical sense. (She and the FOIA office didn&#8217;t respond to a request for comment.)</p><p>There are a few other inconsistencies in the results. Since the numbers seem to conform to whatever the FOIA office wants to argue at that moment, I figure it&#8217;s best not to tell them. I&#8217;m starting to understand the typical response to the crisis in science. Maybe most people figure it&#8217;s best to leave science alone and wait for it to dig itself deeper.</p><p></p>]]></content:encoded></item><item><title><![CDATA[One of the most dangerous ideas in science was a misunderstanding]]></title><description><![CDATA[It's only getting more popular]]></description><link>https://www.redteamofscience.com/p/one-of-the-most-dangerous-ideas-in</link><guid isPermaLink="false">https://www.redteamofscience.com/p/one-of-the-most-dangerous-ideas-in</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Mon, 08 Dec 2025 14:21:43 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!zpNl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The battle for the soul of physics follows a familiar pattern: the winners versus losers. The early replication crisis was winners versus losers too. Established scientists called those who believed in the crisis &#8220;methodological terrorists&#8221; and &#8220;second-stringers.&#8221; Today, winner physicists say that critics are &#8220;<a href="https://www.youtube.com/watch?v=70vYj1KPyT4&amp;t=1860s">poisoning society</a>,&#8221; and claim to be &#8220;<a href="https://youtu.be/miJbW3i9qQc?si=A_cN-YZqPwYsZbcg&amp;t=2234">punching down</a>.&#8221; The winners can&#8217;t help letting you know they&#8217;re winners.</p><p>These physics defenders have lined up against critics who say certain non-empirical theories are nonsense. The defenders claim 1) that these theories are not all of physics, 2) not a lot of physics, and 3) even if they were, it&#8217;s fine because the theories might be true.</p><p>All of these responses focus on the theories, not the glaring issue of empiricism physicists have been elaborately underscoring for decades.</p><p>In 2015, string theorists turned explicitly against evidence. They brought out an intellectual superweapon, something that would not only save their theory but all theories. By the time the philosopher behind this admitted physicists may have misunderstood, the genie was out of the bottle.</p><p>Despite this obvious danger to the scientific bedrock, empiricism gets lost in squabbles over rhetoric: should <em><a href="https://physicsworld.com/a/jesper-grimstrups-the-ant-mill-could-his-anti-string-theory-rant-do-string-theorists-a-favour/">Physics World</a></em> have said &#8220;cult&#8221; or &#8220;cult-like?&#8221; Should they have put &#8220;obedient idiots&#8221; in quotes?<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a> On YouTube, Professor Dave, a science communicator with 4 million followers, <a href="https://www.youtube.com/watch?v=6P_tceoHUH4">escalated the fight</a> well beyond his &#8220;poisoning society&#8221; line to the idea that we need &#8220;<a href="https://youtu.be/6P_tceoHUH4?si=XZYkkkyMZCF0lU9f&amp;t=1820">warriors for science</a>&#8221; in his critique of Sabine Hossenfelder&#8217;s frequent use of &#8220;science is broken&#8221; and &#8220;I don&#8217;t trust scientists.&#8221;</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!SiT2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!SiT2!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png 424w, https://substackcdn.com/image/fetch/$s_!SiT2!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png 848w, https://substackcdn.com/image/fetch/$s_!SiT2!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png 1272w, https://substackcdn.com/image/fetch/$s_!SiT2!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!SiT2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png" width="1358" height="871" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/cf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:871,&quot;width&quot;:1358,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1176224,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/180897388?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!SiT2!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png 424w, https://substackcdn.com/image/fetch/$s_!SiT2!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png 848w, https://substackcdn.com/image/fetch/$s_!SiT2!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png 1272w, https://substackcdn.com/image/fetch/$s_!SiT2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fcf7038a0-53d3-48ce-9392-7eab645dd0bf_1358x871.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Warriors for science</figcaption></figure></div><p>Hossenfelder&#8217;s rhetoric is no doubt blunt, but she&#8217;s made clear that the issue is a lack of attention to empiricism. Before she started, as she admits, &#8220;trying to annoy physics into doing something,&#8221; she wrote the <a href="https://aeon.co/essays/post-empirical-science-is-an-oxymoron-and-it-is-dangerous">article</a>, the <a href="https://www.nature.com/articles/nphys4079">paper</a>, the <a href="https://www.hachettebookgroup.com/titles/sabine-hossenfelder/lost-in-math/9781541646766/?lens=basic-books">book</a>, <a href="https://www.quantamagazine.org/a-fight-for-the-soul-of-science-20151216/">spoke at the conference</a>, and did <a href="https://www.youtube.com/watch?v=KTp1Crg0MY0">boring videos that few watched</a>. The fact is that showing that something is non-empirical in a paper doesn&#8217;t enlist every physicist on Earth to your cause, even though it should. It gets a few dozen citations. What she gets now are longer and <a href="https://www.youtube.com/watch?v=oipI5TQ54tA">longer responses</a> that avoid the issue.</p><p>Here is the simplest form of the argument that can&#8217;t be misunderstood: non-empiricism is an intellectual superweapon. It gives the possessor extraordinary and unearned powers of false discovery. Defending science from it is exhausting because many would like to possess it, and it quickly gets turned on you.</p><p>The debate has become so detached it seems to be about denying the superweapon exists, let alone who&#8217;s using it. Professor Dave isn&#8217;t telling the troops what they&#8217;re fighting for. What he says over and over, in a style that can only be called &#8220;relentless,&#8221; is that defending academia is defending <em>empiricism</em>.</p><p></p><h3>The Misunderstanding</h3><p>In 2015, physicist and philosopher of science Richard Dawid published what would become his most cited <a href="https://iris.unito.it/bitstream/2318/1657963/3/NAA_final.pdf">paper</a>. It rests on the definition of the word &#8220;confirmation,&#8221; which he has rewritten so it looks like a finding:</p><blockquote><p>&#8220;If one measures confirmation in terms of increase in degrees of belief, as Bayesians typically do, T is confirmed by E whenever P(T|E) &gt; P(T). Throughout the paper, we use this inequality as a criterion for when a piece of evidence confirms a hypothesis.&#8221;</p></blockquote><p>Dawid doesn&#8217;t cite anyone here because there&#8217;s no one to cite. The Bayesian epistemologists he cites elsewhere use different definitions and 1. make clear that &#8220;confirms&#8221; means &#8220;is evidence for&#8221; not the usual definition &#8220;verify&#8221; and 2. use &#8220;iff&#8221; instead of &#8220;whenever&#8221;:</p><div class="captioned-image-container"><figure><div class="image-link image2" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Cv1P!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png 424w, https://substackcdn.com/image/fetch/$s_!Cv1P!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png 848w, https://substackcdn.com/image/fetch/$s_!Cv1P!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png 1272w, https://substackcdn.com/image/fetch/$s_!Cv1P!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!Cv1P!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png" width="499" height="165" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/b26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:165,&quot;width&quot;:499,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:44351,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/180897388?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8a1009e8-5f80-4e33-a0ee-4806761c6349_499x170.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Cv1P!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png 424w, https://substackcdn.com/image/fetch/$s_!Cv1P!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png 848w, https://substackcdn.com/image/fetch/$s_!Cv1P!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png 1272w, https://substackcdn.com/image/fetch/$s_!Cv1P!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb26b470e-bf1d-48c4-b313-f56c5974a41f_499x165.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div></div></div></div><figcaption class="image-caption">Previous definition from <a href="http://www.stephanhartmann.org/wp-content/uploads/2014/07/HajekHartmann_BayesEpist.pdf">H&#225;jek and Hartmann (2010)</a></figcaption></figure></div><div class="captioned-image-container"><figure><div class="image-link image2" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!izr2!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png 424w, https://substackcdn.com/image/fetch/$s_!izr2!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png 848w, https://substackcdn.com/image/fetch/$s_!izr2!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png 1272w, https://substackcdn.com/image/fetch/$s_!izr2!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!izr2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png" width="588" height="231" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:231,&quot;width&quot;:588,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:58974,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/180897388?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F47ea43f4-90e4-4497-bc18-4e192c495966_588x231.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!izr2!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png 424w, https://substackcdn.com/image/fetch/$s_!izr2!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png 848w, https://substackcdn.com/image/fetch/$s_!izr2!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png 1272w, https://substackcdn.com/image/fetch/$s_!izr2!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F16ffef3f-2b9b-455d-aa00-2390f6bd12e5_588x231.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div></div></div></div><figcaption class="image-caption">Previous definition from <a href="http://www.stephanhartmann.org/wp-content/uploads/2014/07/HartmannSprenger_BayesEpis.pdf">Hartmann and Sprenger (2010)</a></figcaption></figure></div><p>The use of &#8220;confirms&#8221; is not a finding. It is the definition of a confusing philosophical term. What he&#8217;s doing is a sort of scientific arbitrage, importing a trivial term from philosophy into a field where he knows it means something else. The authors he cites, who are also his co-authors, had no problem making this clear for their readers in philosophy.</p><p>Dawid recycles this definition in a later <a href="https://doi.org/10.1007/s10701-017-0132-1">paper</a> and again ties it to ordinary Bayesians by rewriting Bayes&#8217; theorem:</p><div class="captioned-image-container"><figure><div class="image-link image2" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!L9QP!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png 424w, https://substackcdn.com/image/fetch/$s_!L9QP!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png 848w, https://substackcdn.com/image/fetch/$s_!L9QP!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png 1272w, https://substackcdn.com/image/fetch/$s_!L9QP!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!L9QP!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png" width="326" height="96" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/e73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:96,&quot;width&quot;:326,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:5163,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/180897388?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!L9QP!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png 424w, https://substackcdn.com/image/fetch/$s_!L9QP!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png 848w, https://substackcdn.com/image/fetch/$s_!L9QP!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png 1272w, https://substackcdn.com/image/fetch/$s_!L9QP!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe73e4742-061e-4aa9-8149-ffc0a09e88b7_326x96.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div></div></div></div></figure></div><p>Somehow dividing both sides by P(T) means Bayes&#8217; theorem &#8220;implies that data E confirms H if P(E|T) &gt; P(E).&#8221; This is algebraically equivalent to the definition for the word &#8220;confirms&#8221; above, except he drops the &#8220;if and only if.&#8221;</p><p>Confirmation, as before, can occur even if P(T|E) is near zero, which means that a theory is not worth the paper it&#8217;s printed on.</p><p>But this is just evidence that Dawid knows he&#8217;s guilty. The evidence against the formulation itself is simple. The fact that P(T|E) &gt; P(T) is not confirmation is self-evident. It means you can confirm the moon is made of cheese by its color. The definition is so extreme it is the basis for the famous <a href="https://en.wikipedia.org/wiki/Raven_paradox">Raven Paradox</a>. Dawid doesn&#8217;t cite this or any other dissent.</p><p>Furthermore, ordinary Bayesians would say that if you put one of the 10<sup>500</sup> versions of string theory into Bayes&#8217; theorem, you get something like 1/10<sup>500</sup>, or zero, plus or minus some evidence of which there is almost none. At a conference organized to debate the &#8220;confirms&#8221; paper, Dawid himself said <a href="https://www.youtube.com/watch?v=isQVznEhccY">there is none</a>. According to Bayes&#8217; theorem, zero is a very good estimate.</p><p></p><h3>Benefit of the Doubt</h3><p>Let&#8217;s make a strong case for Dawid, though, and say all he is guilty of is endorsing an extreme form of cherry-picking. Any evidence you can find for something does technically increase the odds it&#8217;s true. One has to admit that other evidence may decrease it, but so what? It doesn&#8217;t matter if he&#8217;s used a word <em>confirmation</em> that might look good on a grant application or in a newspaper. He&#8217;s guilty of fighting for his side with a little wordplay. Let&#8217;s also suppose this society we&#8217;re so afraid of poisoning can survive extreme cherry-picking and claiming any evidence as confirmation. (It can&#8217;t.)</p><p>The really egregious part is that the point of Dawid appealing to the power of evidence is to ditch evidence. This is just one paper in his careerlong argument for non-empirical confirmation: You can accept your theory on the thinnest possible evidence. If you can&#8217;t find any evidence at all, that&#8217;s fine too.</p><p>In a later paper, Dawid cites his own work to endorse &#8220;arguments that resemble empirical confirmation.&#8221; (For logicians, an argument that resembles confirmation but isn&#8217;t confirmation can&#8217;t raise the probability that a theory is true according to that if-and-only-if Dawid left out.)</p><p></p><h3>The Warnings</h3><p>George Ellis, one of the world&#8217;s most prominent physicists, <a href="https://www.nature.com/articles/516321a">warned</a> against Dawid in 2014. His and other non-empirical ideas were called the &#8220;<a href="https://www.npr.org/sections/13.7/2015/01/27/381809832/the-most-dangerous-ideas-in-science">most dangerous ideas in science</a>&#8221; at the time. Here&#8217;s Ellis at that conference doing all that you can do sometimes:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!zpNl!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!zpNl!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg 424w, https://substackcdn.com/image/fetch/$s_!zpNl!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg 848w, https://substackcdn.com/image/fetch/$s_!zpNl!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!zpNl!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!zpNl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg" width="493" height="560" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:560,&quot;width&quot;:493,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:87922,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/180897388?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F7a7e9af3-32b1-43fc-a465-3bc2cfcdeb10_1000x560.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!zpNl!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg 424w, https://substackcdn.com/image/fetch/$s_!zpNl!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg 848w, https://substackcdn.com/image/fetch/$s_!zpNl!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!zpNl!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F99b67469-031d-4a53-8988-e4780e6e2af8_493x560.jpeg 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">George Ellis in 2015 at a conference debating empiricism</figcaption></figure></div><p>After the conference, Ellis claimed the same victory, saying that string theorists had admitted that the theory wasn&#8217;t &#8220;&#8217;confirmed&#8217; in the sense of being verified.&#8221; Two years later, Dawid <a href="https://arxiv.org/pdf/1702.01133">responded in the literature</a>, stating conclusively that there&#8217;s an &#8220;unfortunate mismatch&#8221; in the way the term is used in philosophy compared to physics, which may lead to some misunderstanding.</p><p></p><h3>The Future</h3><p>None of this stopped the paper from being <a href="https://scholar.google.com/citations?view_op=view_citation&amp;hl=en&amp;user=86Rb4fwAAAAJ&amp;citation_for_view=86Rb4fwAAAAJ:Wp0gIr-vW9MC">cited</a>. The &#8220;no alternatives argument&#8221; is still being used today, ironically to show that there isn&#8217;t a crisis and the critics are just <a href="https://physicsworld.com/a/jesper-grimstrups-the-ant-mill-could-his-anti-string-theory-rant-do-string-theorists-a-favour/">sore losers</a>.</p><div class="captioned-image-container"><figure><div class="image-link image2" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!LYKg!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png 424w, https://substackcdn.com/image/fetch/$s_!LYKg!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png 848w, https://substackcdn.com/image/fetch/$s_!LYKg!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png 1272w, https://substackcdn.com/image/fetch/$s_!LYKg!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!LYKg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png" width="640" height="125" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:125,&quot;width&quot;:640,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:31490,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/180897388?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Ff19829c5-4981-41b8-a590-e75a60ef8375_640x125.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!LYKg!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png 424w, https://substackcdn.com/image/fetch/$s_!LYKg!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png 848w, https://substackcdn.com/image/fetch/$s_!LYKg!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png 1272w, https://substackcdn.com/image/fetch/$s_!LYKg!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F460170e2-a6e0-4371-9045-33c5bda8272a_640x125.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div></div></div></div><figcaption class="image-caption">Physics World (2025)</figcaption></figure></div><p>Sour grapes, as Professor Dave would say.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!NQBD!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!NQBD!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png 424w, https://substackcdn.com/image/fetch/$s_!NQBD!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png 848w, https://substackcdn.com/image/fetch/$s_!NQBD!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png 1272w, https://substackcdn.com/image/fetch/$s_!NQBD!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!NQBD!,w_2400,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png" width="1200" height="446.7032967032967" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:false,&quot;imageSize&quot;:&quot;large&quot;,&quot;height&quot;:542,&quot;width&quot;:1456,&quot;resizeWidth&quot;:1200,&quot;bytes&quot;:3920831,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/180897388?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:&quot;center&quot;,&quot;offset&quot;:false}" class="sizing-large" alt="" srcset="https://substackcdn.com/image/fetch/$s_!NQBD!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png 424w, https://substackcdn.com/image/fetch/$s_!NQBD!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png 848w, https://substackcdn.com/image/fetch/$s_!NQBD!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png 1272w, https://substackcdn.com/image/fetch/$s_!NQBD!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F2ca27db2-0a3a-409d-b997-49ea225a40c1_3993x1486.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Some Professor Dave explanations</figcaption></figure></div><p>There&#8217;s a much larger audience on YouTube than <em>Physics World</em> could dream of, where Professor Dave defends string theory as empirical. He might be benefiting from the superweapon unwittingly, who knows. But he seems to think the issue is as serious as it gets. The question is whether or not he can be convinced that he&#8217;s the one with the bomb.</p><p></p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>This was, I now realize, an unfortunate saga in which I participated by emailing the editor and asking for a response to the review from an empiricist. Physics World has updated the <a href="https://physicsworld.com/a/jesper-grimstrups-the-ant-mill-could-his-anti-string-theory-rant-do-string-theorists-a-favour/">review</a> by linking to a response from Grimstrup and making (undocumented) changes to the text. That said, I think the Grimstrup post is thin-skinned to the point of censoriousness. In my opinion, everyone lost sight of the stakes involved.</p></div></div>]]></content:encoded></item><item><title><![CDATA[Psychologists think there's a crisis in psychology]]></title><description><![CDATA[Results from a new poll]]></description><link>https://www.redteamofscience.com/p/psychologists-think-theres-a-crisis</link><guid isPermaLink="false">https://www.redteamofscience.com/p/psychologists-think-theres-a-crisis</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Mon, 24 Nov 2025 12:03:44 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!39xq!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Metascience principles are now <a href="https://www.science.org/doi/10.1126/science.adz9553">law</a> in the United States. The law drew <a href="https://www.cos.io/about/news/cos-statement-on-restoring-gold-standard-science-executive-order">protests</a> from the field, but metascience is undoubtedly influential, and it has the largest purview of any scientific field. Yet the huge questions metascience needs to answer have been left to a few scattered groups.</p><p>Two of these questions in psychology were probed in 2016 and never repeated in such an explicit way since: &#8220;Are we in a replication crisis?&#8221; and &#8220;If so, why?&#8221;<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a></p><p>Spencer Greenberg&#8217;s group at Clearer Thinking <a href="https://replications.clearerthinking.org/how-much-do-academic-psychologists-trust-academic-psychology-and-is-there-still-a-replication-crisis">published results of a survey</a> a few weeks ago that helps to end this drought and may help us know if anything is working. They asked psychologists if psychology is still in crisis and why.</p><p>Clearer Thinking has been picking up academia&#8217;s slack in several ways, notably rapid replication, and proposing solutions and hypotheses too impolite towards academics to have come out of academia. These are not outlandish or overly insulting ideas. For instance, the proposal that scientific reforms might act as patches to a <a href="https://www.clearerthinking.org/post/importance-hacking-a-major-yet-rarely-discussed-problem-in-science">leaky pipe</a> that causes greater pressure on other cracks, or that replications should be done soon after publication to incentivize researchers to practice greater rigor. These are good ideas but they are too impolite. That is the background on Clearer Thinking, and academia lobbing itself softballs happens to be the whole problem in a nutshell.</p><p>The problem with Clearer Thinking is they are still a little bit polite, which I hope to correct here.</p><p></p><h3>Are we in a crisis?</h3><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Pt0K!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Pt0K!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png 424w, https://substackcdn.com/image/fetch/$s_!Pt0K!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png 848w, https://substackcdn.com/image/fetch/$s_!Pt0K!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png 1272w, https://substackcdn.com/image/fetch/$s_!Pt0K!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Pt0K!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png" width="1024" height="586" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:586,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:29191,&quot;alt&quot;:&quot;&quot;,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/179571952?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" title="" srcset="https://substackcdn.com/image/fetch/$s_!Pt0K!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png 424w, https://substackcdn.com/image/fetch/$s_!Pt0K!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png 848w, https://substackcdn.com/image/fetch/$s_!Pt0K!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png 1272w, https://substackcdn.com/image/fetch/$s_!Pt0K!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa4139fe1-5697-4872-b4c2-e5a11cfc34ca_1024x586.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Reproduced from clearerthinking.org</figcaption></figure></div><p>The poll almost perfectly agrees with Monya Baker&#8217;s 2016 <a href="https://www.nature.com/articles/533452a">poll in Nature</a>, except most respondents now say &#8220;substantial progress has been made.&#8221; Baker&#8217;s poll found 90% of researchers believed there was a reproducibility crisis in science, and 52% said it was significant. The psychologists in the poll agreed (n = 44) at a slightly higher rate. In other words, 93% of psychologists would have used the word &#8220;crisis&#8221; then, and 95% would use it now except that most now believe a lot of progress has been made nonetheless.</p><p></p><h3>What are we doing about the crisis?</h3><p>The poll asks what improvements participants have made in their own research, and the ones who have made changes generally adopted preregistration (61%). So if we&#8217;re looking for progress, the best place to start might be preregistration.</p><p>As the survey suggests, preregistration is a clear and obvious winner among scientific reforms. It addresses base rate neglect, p-hacking, HARKing, and publication bias. Preregistration is backed by 2,000 years of thought on post hoc reasoning. It&#8217;s backed by statistics and something better than statistics: logic. Preregistration is already part of what protects us from dangerous drugs and medical devices. However, if preregistration adoption is the significant progress these psychologists are citing, it has not shown up in any studies of the literature.</p><p>The percentage of papers in psychology that are preregistered was <a href="https://doi.org/10.1177/25152459241283477">measured</a> in the early 2010s and again in 2022 and it rose from 2% to 7%. This is significant when you consider the intransigence of scientific practice and its size, the billions of dollars wrapped up in publishing and so on. However, a 5% increase doesn&#8217;t substantially move the replication rate.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-2" href="#footnote-2" target="_self">2</a></p><p>Much more importantly, the slow adoption of preregistration shows <em>what type of problem scientific reform is</em>. It shows 21st century reform isn&#8217;t the kind of thing that just catches on.</p><p>Compare preregistration with a practice that slightly <em>increases</em> one&#8217;s chances of publication, the Benjamini-Hochberg procedure, which has been cited <a href="https://scholar.google.com/scholar?hl=en&amp;as_sdt=0%2C20&amp;q=+Benjamini+Y%2C+Hochberg+Y+%281995%29.+%22Controlling+the+false+discovery+rate%3A+a+practical+and+powerful+approach+to+multiple+testing%22.&amp;btnG=">120,000 times</a>. There are no non-profits set up to encourage use of the Benjamini-Hochberg over the less forgiving Bonferroni. Adoption happened automatically. 21st century reforms are not like this. They&#8217;re spinach, not ice cream.</p><p>One of the most important questions in the epistemic life of Earth&#8217;s 8 billion people is: Are scientists going to adopt reform or are they going to resist? This is the question we need answered. We need to interpret what a 5% increase in adoption &#8212; maybe 6 or 7% by now &#8212; means.</p><p>Change that reduces publication, even in metascience&#8217;s home turf of psychology, is spinach. We know it is. Even the small amount of progress is among self-selected adopters. They could be the ones who can afford to preregister. Their work is more certain to start with. Their subfield is more vague. Their job less in doubt. It moves the needle of discovery much less than its own prevalence might suggest, and it doesn&#8217;t predict future adoption.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!39xq!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!39xq!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png 424w, https://substackcdn.com/image/fetch/$s_!39xq!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png 848w, https://substackcdn.com/image/fetch/$s_!39xq!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png 1272w, https://substackcdn.com/image/fetch/$s_!39xq!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!39xq!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png" width="1024" height="1024" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1024,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1975249,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/179571952?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!39xq!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png 424w, https://substackcdn.com/image/fetch/$s_!39xq!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png 848w, https://substackcdn.com/image/fetch/$s_!39xq!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png 1272w, https://substackcdn.com/image/fetch/$s_!39xq!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F66dddb8a-f1a5-4bca-95c1-bfd5af13e35e_1024x1024.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p></p><h3>The status of p-hacking in 2025</h3><p>The survey asked how serious p-hacking is at the top 5 psychology journals. The results look like this:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!e8fr!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!e8fr!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png 424w, https://substackcdn.com/image/fetch/$s_!e8fr!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png 848w, https://substackcdn.com/image/fetch/$s_!e8fr!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png 1272w, https://substackcdn.com/image/fetch/$s_!e8fr!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!e8fr!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png" width="837" height="831" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/10be877b-0938-4559-9c47-356a62e78097_837x831.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:831,&quot;width&quot;:837,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:28788,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/179571952?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!e8fr!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png 424w, https://substackcdn.com/image/fetch/$s_!e8fr!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png 848w, https://substackcdn.com/image/fetch/$s_!e8fr!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png 1272w, https://substackcdn.com/image/fetch/$s_!e8fr!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F10be877b-0938-4559-9c47-356a62e78097_837x831.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Severity of p-hacking at respondents&#8217; top five journals (data: clearerthinking.org)</figcaption></figure></div><p></p><p>To put &#8220;top 5&#8221; in perspective, the top journals occupy a hierarchy that is shaped like other highly competitive environments:</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!zCht!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!zCht!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png 424w, https://substackcdn.com/image/fetch/$s_!zCht!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png 848w, https://substackcdn.com/image/fetch/$s_!zCht!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png 1272w, https://substackcdn.com/image/fetch/$s_!zCht!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!zCht!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png" width="832" height="830" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a8ad335b-0d08-4796-b98e-43aead077b69_832x830.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:830,&quot;width&quot;:832,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:15108,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/179571952?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!zCht!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png 424w, https://substackcdn.com/image/fetch/$s_!zCht!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png 848w, https://substackcdn.com/image/fetch/$s_!zCht!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png 1272w, https://substackcdn.com/image/fetch/$s_!zCht!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fa8ad335b-0d08-4796-b98e-43aead077b69_832x830.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">Distribution of impact factors at psychology journals (data: journalimpact.org)</figcaption></figure></div><p>Only a few are elite and many exponentially less so. Impact factor is not a good measure but it does measure influence. If p-hacking is a problem in the elite, its findings are at least influential there. It also stands to reason that it&#8217;s probably a problem everywhere.</p><p></p><h3>Why do researchers bend the rules?</h3><p>Famously, early reformers in psychology blamed the unconscious and forgetfulness. Feynman blamed &#8220;fooling yourself.&#8221; These charitable explanations are certainly partially true but they are starting to lose steam as predictors. If being charitable was supposed to be a catalyst, it is failing at that too.</p><p>In the free-form responses to the &#8220;reforms adopted&#8221; question, nobody mentioned combating their unconscious or forgetfulness because nobody believes those hypotheses. Nobody has bothered to find out if they are correct because they&#8217;re obviously not. A better predictor consistent with available evidence is <a href="https://doi.org/10.1098/rsos.160384">Darwinian</a>: Academics are p-hacking for survival in their profession and in their life&#8217;s work. They are doing so most likely on purpose or in deep denial. If they didn&#8217;t bend the rules, they might not have a job and all their friends, loved ones, and the rest of science would blithely continue on without them, winning Nobel prizes, educating students, and truly discovering some things. Discovering less maybe, but it&#8217;s the price of survival.</p><p></p><h3>The replication rate</h3><p>In another indication that 95% of psychologists are right that there still is a slight crisis, participants guessed the replication rate for the top 5 psychology journals and the average guess was 55%. We&#8217;re used to this &#8776;50% metric now, but in <a href="https://doi.org/10.1371/journal.pmed.0020124">2005</a> it was shocking. After all of this progress, the participants&#8217; favorite journals are still only around a coin flip. Hopefully progress is not just inuring us to the numbers.</p><p>One optimistic way to interpret the higher adoption of reforms in some top journals (as high as 58% by <a href="https://datacolada.org/115">one count</a>) is to say that true discoveries will come out of better journals. The total replication rate doesn&#8217;t matter as long as some good work has been done and it shades out the rest. This is truly hopeful and it is possible. Unfortunately, the data didn&#8217;t indicate elite work either. Respondents were asked which findings have held up over the last 15 years. Other than a small cluster who said metascience, and a larger one who said there aren&#8217;t any, every answer was different.</p><p></p><h3>Responses:</h3><p>There&#8217;s much more in the data and Greenberg says there will be more analyses in the next few months. Announcements of new posts come through their <a href="https://replications.clearerthinking.org/subscribe/">newsletter</a>. Their <a href="https://replications.clearerthinking.org/how-much-do-academic-psychologists-trust-academic-psychology-and-is-there-still-a-replication-crisis">post</a> was by Director of Replications Amanda Metskas. I clarified the replication estimates came from participants after a review by Clearer Thinking.</p><p>Both Baker and Clearer Thinking shared data. The code for this post is <a href="https://github.com/alexbyrnes/clearer_thinking_reanalysis">here</a>.</p><p></p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>For a similar poll in biomedicine, see <a href="https://doi.org/10.1371/journal.pbio.3002870">Cobey et al., 2024</a>.</p></div></div><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-2" href="#footnote-anchor-2" class="footnote-number" contenteditable="false" target="_self">2</a><div class="footnote-content"><p>The Registered Report format is less hackable than preregistration and arguably more supportive to researchers&#8217; careers. Its <a href="https://doi.org/10.1162/qss_a_00364">prevalence is around 2.7%</a> in its best year (2023) despite near unanimous endorsement in the community and among community leaders.</p><p></p></div></div>]]></content:encoded></item><item><title><![CDATA[Cognitive dissonance and a truly impossible assumption]]></title><description><![CDATA[The many unopened boxes of psychology]]></description><link>https://www.redteamofscience.com/p/cognitive-dissonance-and-a-truly</link><guid isPermaLink="false">https://www.redteamofscience.com/p/cognitive-dissonance-and-a-truly</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Wed, 12 Nov 2025 15:16:07 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/fefa9e42-7a34-4f26-8369-2522efbe9dfa_1362x1362.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>One reaction to <a href="https://doi.org/10.1002/jhbs.70043">news</a> that another keystone psychology finding has been severely limited by researcher dishonesty would be to wonder if there&#8217;s anything left. Is there anything to psychology at all? Another would be the opposite, that we need to stop throwing out whole fields, that &#8220;the replication crisis brain&#8221; is a totalizing <em>epistemic black hole</em>.</p><div class="captioned-image-container"><figure><a class="image-link image2" target="_blank" href="https://bsky.app/profile/devezer.bsky.social/post/3m4y6ehyzws2q" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Cr90!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png 424w, https://substackcdn.com/image/fetch/$s_!Cr90!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png 848w, https://substackcdn.com/image/fetch/$s_!Cr90!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png 1272w, https://substackcdn.com/image/fetch/$s_!Cr90!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Cr90!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png" width="597" height="157" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/c3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:157,&quot;width&quot;:597,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:32633,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:&quot;https://bsky.app/profile/devezer.bsky.social/post/3m4y6ehyzws2q&quot;,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/178597140?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Cr90!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png 424w, https://substackcdn.com/image/fetch/$s_!Cr90!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png 848w, https://substackcdn.com/image/fetch/$s_!Cr90!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png 1272w, https://substackcdn.com/image/fetch/$s_!Cr90!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc3cc8291-0252-43c9-8a93-c69d566ae470_597x157.png 1456w" sizes="100vw" fetchpriority="high"></picture><div></div></div></a></figure></div><p>Another would be to doubt the veracity of the news. It makes sense to doubt that a single independent researcher without apparent credentials could overturn 70 years of a scientific discipline by opening an unsealed archive called &#8220;box 4.&#8221;</p><p>Surprisingly, psychologists don&#8217;t seem to be doubting the source. The source is <em>The Journal of the History of the Behavioral Sciences</em> and independent researcher Thomas Kelly who claims that the seminal <em>When Prophecy Fails</em> from the 1950s is bunk.</p><p><em>When Prophecy Fails</em> is about cult leaders and <em>cognitive dissonance</em> from which they suffered. Kelly is daring to say a cult wasn&#8217;t as wrong about something and researchers who infiltrated it were more so. Surely he could be doubted. The debate is the efforts of a major scientific discipline over 70 years versus a guy with a box. Yet even psychology&#8217;s defenders don&#8217;t want to bet against the box.</p><p>If you&#8217;ve ever had vast swathes of your life decided by psychology, or watched some of the great questions of the day weighed by psychologists, or have sympathy for people with skin in the game so to speak, you might feel angry. Your reaction might not be scientific. You might not <em>demand something be measured</em>.</p><p>You would be wrong, though. The thing to measure is the thing itself: how much scientists lie or are consciously biased. Kelly takes the rare path and states it: &#8220;the researchers&#8217; notes leave no option but to conclude the misrepresentations were intentional.&#8221;</p><p>As he suggests, the possibility of intention has a mathematical quality to it now. There doesn&#8217;t seem to be any option left but to consider it.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!i0-w!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!i0-w!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png 424w, https://substackcdn.com/image/fetch/$s_!i0-w!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png 848w, https://substackcdn.com/image/fetch/$s_!i0-w!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png 1272w, https://substackcdn.com/image/fetch/$s_!i0-w!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!i0-w!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png" width="1362" height="1362" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1362,&quot;width&quot;:1362,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:476496,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/178597140?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!i0-w!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png 424w, https://substackcdn.com/image/fetch/$s_!i0-w!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png 848w, https://substackcdn.com/image/fetch/$s_!i0-w!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png 1272w, https://substackcdn.com/image/fetch/$s_!i0-w!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6de23a6e-f78e-4ef5-9a4c-5d95aade038a_1362x1362.png 1456w" sizes="100vw"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>Unfortunately we&#8217;re very polite about lying in science. So much so that it took an anonymous researcher reading 2,500 papers to finally say, &#8220;<a href="https://www.fantasticanachronism.com/p/whats-wrong-with-social-science-and-how-to-fix-it">Stop assuming good faith</a>&#8221; in 2020.</p><p>Metaresearchers &#8212; that is, people who study other researchers &#8212; leave out intentionality except in extreme cases. This year, a <a href="https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-025-02590-6">paper</a> considered only two possibilities on the topic: innocent bias, and deliberate misconduct like making up data. Deliberate use of the grey area is, and has always been, <a href="https://www.sensible-med.com/p/non-random-and-intentional-errors">unmentioned and unmeasured</a>. P-values <a href="https://journals.sagepub.com/doi/10.1177/25152459251323480">move on their own</a>. If the agent is involved at all, bias is due to forgetfulness or the unconscious.</p><p>None of the psychologists who said unconscious bias is a major factor in the replication crisis have updated their beliefs over 15 years. Yet p-hacking is obviously more conscious after one knows what p-hacking is, otherwise there&#8217;d be no point to discussing it. But psychologists still hang on to the unconscious explanation.</p><p>At least we can state the very obvious fact that we don&#8217;t know if questionable research practices are mostly intentional or not. We at least don&#8217;t know because we can&#8217;t diagnose at a distance. This goes unstated and it&#8217;s trivial to anyone outside.</p><p>The easy answer as to why reformers don&#8217;t want to make this rude suggestion about their colleagues is that they are biased too and want to preserve their place in academia like anyone else. A harder answer to face is that the measurement of lies is volatile. Once it&#8217;s out there, &#8220;researchers are lying&#8221; can be used to support any conclusion you like. &#8220;Researchers are lying&#8221; could be the mantra from the lowly extremist to the highest reaches of our politically powerful, including the most moderate meta-researcher in between. We might not want to know what&#8217;s in that box.</p><p>The more modern take, though, is consistent with the available evidence that nobody is surprised anymore. Maybe we already know what&#8217;s in the box.</p><p></p><h5><em>Responses:</em></h5><p><em>Both Devezer and Kelly responded with brief notes, which I appreciate very much. I changed two misleading words and linked to the <a href="https://bsky.app/profile/devezer.bsky.social/post/3m4y2ilm63k2j">full thread</a> from Devezer for context.</em></p>]]></content:encoded></item><item><title><![CDATA[Physics defends itself]]></title><description><![CDATA[There's not as much math as you'd expect]]></description><link>https://www.redteamofscience.com/p/physics-defends-itself</link><guid isPermaLink="false">https://www.redteamofscience.com/p/physics-defends-itself</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Mon, 03 Nov 2025 13:10:51 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/bef62540-778c-4ab7-a751-79c909510798_3256x754.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In philosophy of science, coming up with just the right band-aid that rescues a hypothesis and makes no predictions of its own is considered highly suspicious and the worst kind of what&#8217;s called &#8220;accommodation.&#8221; Ad hominem attacks are an even lesser response. Insults have to rank lower still.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!wCET!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!wCET!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png 424w, https://substackcdn.com/image/fetch/$s_!wCET!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png 848w, https://substackcdn.com/image/fetch/$s_!wCET!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png 1272w, https://substackcdn.com/image/fetch/$s_!wCET!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!wCET!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png" width="1224" height="567" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/d0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:567,&quot;width&quot;:1224,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:195202,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/177835633?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!wCET!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png 424w, https://substackcdn.com/image/fetch/$s_!wCET!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png 848w, https://substackcdn.com/image/fetch/$s_!wCET!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png 1272w, https://substackcdn.com/image/fetch/$s_!wCET!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd0eedca3-e334-4e19-b640-d835fd2ec78f_1224x567.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>In 2016, a prominent editor and social psychologist Susan Fiske called metascientists &#8220;methodological terrorists.&#8221; Statistician Andrew Gelman wrote a <a href="https://statmodeling.stat.columbia.edu/2016/09/21/what-has-happened-down-here-is-the-winds-have-changed/">response</a> on his blog that became a classic. To onlookers, the importance of Dr. Fiske&#8217;s post wasn&#8217;t just that she was being hostile and ad hominem. It was an admission that psychology didn&#8217;t have a better defense against the replication crisis that was having its first empirical confirmations.</p><p>It is therefore interesting to try to understand the defense that physics has put up against a major threat to its credibility, the &#8220;String Theory Wars.&#8221; Works in this genre of critique include <em><a href="http://www.jimbaggott.com/books/farewell-to-reality/">Farewell to Reality</a></em>, <em><a href="https://en.wikipedia.org/wiki/The_Trouble_with_Physics">The Trouble with Physics</a></em>, <em><a href="https://www.hachettebookgroup.com/titles/sabine-hossenfelder/lost-in-math/9781541646766/?lens=basic-books">Lost in Math</a></em>, and recently, <em><a href="https://www.amazon.com/ANT-MILL-theoretical-high-energy-mass-production/dp/8797286893">The Ant Mill</a></em>. These are all base-rate books. Their authors doubt certain theories in part because there are trillions of variations on them and, without empirical evidence to promote one variation over another, the probability that a particular variation is true is near zero.</p><p>The threat to physics&#8217; credibility is, in other words, Bayesianism, a threat that all of academia has been fighting off since the 1800s. It is unlikely that physics has a good argument against Bayesians because no science has had a good argument. It&#8217;s unlikely that physics has been dealing with it internally because no science has.</p><p>Bayesianism means being able to publish less. It admits the degree to which results are subjective. It admits the tenuousness of expertise. These are all things that academics don&#8217;t like.</p><p></p><h3>Physics is not string theory</h3><p>There&#8217;s a long tradition of debate between physicists who believe string theory and other non-empirical and implausible theories should taint physics as a whole, and physicists who think string theory is alone and on the fringe.</p><p>What&#8217;s dismaying to onlookers is that a lot of mainstream physics&#8217; defense against critique has been ad hominem. It has not stuck to the truth of string theory, or whether or not more of physics is also untrue. This is the <a href="https://www.youtube.com/watch?v=kya_LXa_y1E">history of the debate</a>, to paraphrase physicist Angela Collier:</p><ul><li><p>Some string theorists wrote popular science books for 30 years.</p></li><li><p>The great success of string theory these books portrayed was a &#8220;lie.&#8221;</p></li><li><p>In the early 2000s through to the present day, other physicists wrote books to counter this lie but they also tainted physics as a whole.</p></li><li><p>The public stopped wanting to support physics because of this and now &#8220;we have to earn their trust back and that sucks.&#8221;</p></li></ul><p>After Dr. Collier summarized the debate in this way, her YouTube channel was flooded with commenters she called &#8220;physics haters.&#8221; She took great umbrage at this, and in September, <a href="https://www.youtube.com/watch?v=miJbW3i9qQc">made an argument</a> that physics should not be put in the same bucket with the lie.</p><p>Dr. Collier&#8217;s argument was an extreme form of the bad apples defense. When someone loses credibility, science is free to disavow them and claim they were a bad apple, and it often does. The public is left to sort through representatives of science rather than scientific facts.</p><p>Dr. Collier makes matters worse by wading in, particularly on the &#8220;melon heads&#8221; who visited her channel. She calls these alleged physics haters &#8220;stupid,&#8221; &#8220;pathetic,&#8221; and listening to opposing physicists like Sabine Hossenfelder &#8220;embarrassing&#8221; over a dozen times.</p><p>The irony is that these commenters don&#8217;t represent all of the public either, and just because some used absolutist language like &#8220;physics is broken&#8221; doesn&#8217;t mean there isn&#8217;t a more nuanced point to be made. It has been made by physicists for decades. Also, the public generally can&#8217;t distinguish her good topics like antimatter from string theory or anything else. For all the vast majority of the public knows, they could be accused of being stupid and pathetic for watching Angela Collier in a few years.</p><p>The only solution is to try to debate the probability of whether or not string theory and the rest of physics is true. One way to do that, as the authors above have, is to count the number of possible hypotheses in string theory, a number first <a href="https://ned.ipac.caltech.edu/level5/March05/Guth/Guth4.html">published</a> in 2005, and argue that this means that string theory is not true. This is the same argument that was successful in Genome-Wide Association Studies and in <em><a href="https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124">Why most published research findings are false</a></em>. It is an argument the public can understand and the equation is easy to prove.</p><p>The question is, if this argument is credible enough for physicists to tell us, why isn&#8217;t it credible enough for them to tell each other so we don&#8217;t need to get involved?</p><p>The answer is simple. Bayesianism is how physicists think. Frequentism is how they publish. Frequentism keeps the lights on.</p><p></p><h3>The insults</h3><p>Philosopher Robert Crease wrote a <a href="https://physicsworld.com/a/jesper-grimstrups-the-ant-mill-could-his-anti-string-theory-rant-do-string-theorists-a-favour/">review</a> of <em>The Ant Mill</em> last month in <em>Physics World</em> that is quintessentially insulting in a way that would make a youtuber blush. He compares the author, Jesper Grimstrup, to a jilted ex, one you barely remember but has written a whole book about you.</p><p>The &#8220;ex&#8221; part comes from Grimstrup&#8217;s departure from academia, what Crease calls a &#8220;bad breakup.&#8221;</p><p>Academia shouldn&#8217;t listen to its ex Grimstrup, the review says. Listen to current academic Richard Dawid, a string theory supporter. What would the ex analogy be without a new beau?</p><p>String theory opponents have criticized its lack of falsifiability, the domain of perhaps the 20th century&#8217;s greatest philosopher of science, Karl Popper. Academia&#8217;s new beau doesn&#8217;t make elaborate demands like Popper and Grimstrup. He accepts the way things are and simply writes them down: </p><p>&#8220;Dawid, you see, is making the formalism follow the practice rather than the other way around.&#8221;</p><p>Dawid himself isn&#8217;t as quotable as string theorists may like. He admits string theorists have to &#8220;<a href="https://link.springer.com/article/10.1007/s10701-011-9592-x">make the best of it</a>&#8221; in a &#8220;drought&#8221; of empirical evidence. Where paradigm shifts were once due to great empirical confirmations, they are now brought on by lack of evidence.</p><p>Critiquing Popper is not new. String theorist Leonard Susskind called critics &#8220;Popperazzi.&#8221;</p><p>Going after falsification is also a supremely poor accommodation. If you can overturn the greatest philosopher of science of the 20th century, you should publish it, then go back to physics, not wait until the cards are on the table.</p><p>Along with the insults, it&#8217;s telling how much these academics describe their own bodies in space, how it felt to read the physics haters. Collier describes how it &#8220;feels bad&#8221; continually. Crease ends up deciding that The Ant Mill is so &#8220;over-the-top&#8221; that he finds reading it a &#8220;hoot.&#8221;</p><p>How they felt is not even good evidence for them, let alone for us.</p><p></p><h3>Why use the word &#8220;crisis?&#8221;</h3><p>String theory is not all of physics. It is not even a lot percentage-wise. However, the argument for crisis doesn&#8217;t solely rest on how much of physics is nonsense because string theory is mixed in. It rests on the fact that <em>all fields</em> have largely rejected Bayesianism.</p><p>Bayesian physicist Giulio D&#8217;Agostini published a <a href="https://ned.ipac.caltech.edu/level5/March01/Dagostini/Dagostini5.html">paper</a> in 1998, well before Bayesianism became <a href="https://www.economist.com/science-and-technology/2005/09/01/and-statistics">news in 2005</a>. He describes a recent discovery claim in physics. Although the discovery had a tiny p-value, he couldn&#8217;t get any of his colleagues to stake even odds that it was true.</p><p>This means that, despite the finding being publishable, experts didn&#8217;t think there was even a 50% chance it was true. It was clear to D&#8217;Agostini that physicists believed in Bayes&#8217; theorem but didn&#8217;t use it when publishing.</p><p>Predictions are good evidence. Unfortunately, we can&#8217;t go back and make D&#8217;Agostini&#8217;s paper into a famous prediction. It was certainly predictive, though. &#8220;Less than 50%&#8221; became the <a href="https://www.redteamofscience.com/p/science-reform-do-we-need-a-plan">predominant finding</a> for 20 years. In physics, experts being secret Bayesians predicts that the base rate for a prominent theory like string theory might be calculated, which would further confirm to physicists that it&#8217;s not true. It predicts that other physicists wouldn&#8217;t be able to make a Bayesian argument against the theory because they regularly doubt findings behind the public&#8217;s back. Making a Bayesian argument would sink all kinds of ordinary physics papers too.</p><p>With no good alternatives, some physicists would spend 20 years writing books for the public instead.</p><p>The other reason to use the word &#8220;crisis&#8221; is self-explanatory. Getting rid of empiricism lets in a lot of things we&#8217;d rather keep out of science like religion, mysticism, and quackery. Given physics&#8217; place in science and its obvious ability to inspire the foundations of scientific philosophy, continued empiricism during the drought seems like the least we could ask.</p>]]></content:encoded></item><item><title><![CDATA[Physics has all the ingredients for scientific crisis]]></title><description><![CDATA[It turns out all you need is money and a googol of hypotheses to write up]]></description><link>https://www.redteamofscience.com/p/physics-has-all-the-ingredients-for</link><guid isPermaLink="false">https://www.redteamofscience.com/p/physics-has-all-the-ingredients-for</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Mon, 20 Oct 2025 12:02:33 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!IL_T!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In 2016, a famous survey was published in Nature called &#8220;<a href="https://www.nature.com/articles/533452a">1,500 scientists lift the lid on reproducibility</a>.&#8221; It appeared to confirm that scientists themselves agreed there was a &#8220;significant crisis&#8221; in science.</p><p>What had been theory in the 2000s and confirmed empirically in the 2010s was now accepted by most researchers. It turned out they knew partially from their own experience that it was true and worrying. The loop had been closed, it seemed.</p><p>The &#8220;hard&#8221; fields of physics, chemistry, and engineering fared best in the poll. Now, thanks to physicists Sabine Hossenfelder and Jesper M&#248;ller Grimstrup, we may need to start rethinking what became a popular idea, that the crisis is due to the &#8220;softness&#8221; of social psychology (and, for some reason, several other fields).</p><p>Reading Hossenfelder and Grimstrup&#8217;s books <em><a href="https://www.hachettebookgroup.com/titles/sabine-hossenfelder/lost-in-math/9781541646766/?lens=basic-books">Lost in Math</a></em> and <em><a href="https://www.amazon.com/dp/8797286893">The Ant Mill</a></em> is a delight and there&#8217;s real metascience in them. The crisis in physics is a natural experiment in which many of the other potential causes of the crisis from other fields have been removed, leaving only the pure passion for publication and funding.</p><p>In physics, you take away the questions of validity and flexible interpretation from psychology. You take away the closed data and incomplete methods from cancer biology and others. (The existential threat of not getting along with your mentors and co-authors and the other familiar pathologies of science are still there.)</p><p>This experiment suggests that there&#8217;s one common factor in science, the fact that the people who support you have no idea what you&#8217;re doing and you have little incentive to tell them.</p><p>Both authors have made an argument the public may have a chance of understanding, that hypotheses chosen from a near infinite set with nothing to recommend them have no chance of being true. Grimstrup estimates there are 10<sup>500</sup> possible hypotheses in string theory. Each can be a paper.</p><p>It turns out, according to these very detailed accounts, that physicists will happily milk string theory just as psychologists invented new primes.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!IL_T!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!IL_T!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png 424w, https://substackcdn.com/image/fetch/$s_!IL_T!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png 848w, https://substackcdn.com/image/fetch/$s_!IL_T!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png 1272w, https://substackcdn.com/image/fetch/$s_!IL_T!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!IL_T!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png" width="940" height="940" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:940,&quot;width&quot;:940,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1012461,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/176569398?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!IL_T!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png 424w, https://substackcdn.com/image/fetch/$s_!IL_T!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png 848w, https://substackcdn.com/image/fetch/$s_!IL_T!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png 1272w, https://substackcdn.com/image/fetch/$s_!IL_T!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F12e11d37-b9ea-4053-93c3-f8787aaae5fa_940x940.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p></p><h3>Familiar patterns</h3><p>Much of the observations in these books are familiar. They have been observed in other fields and by other reformers. Goodhart&#8217;s Law, the Matthew Effect, a disturbing element of evolution in science towards worse practices.</p><p>Some observations are refreshing that you don&#8217;t often see, certainly not in psychology. The idea that researchers should be judged not only by the compromises they make for the sake of their careers, but for their silence in the face of such an obvious elephant in the room. Grimstrup even calls for &#8212; nondestructive &#8212; reform from outside. Hossenfelder has done more than her share on that front.</p><p>These are, in other words, ordinary observations from an extraordinary place few understand and few would suspect.</p><p>Physicist Alan Sokal once derided the encroachment on academia by those who deny the existence of objective reality. Years later, physicists started to deny reality too. Hossenfelder&#8217;s conversation with the eminent cosmologist George Ellis in which he laments this development, that some physicists are claiming theories don&#8217;t need to be supported by experiment, and feels powerless to do anything about it, is particularly affecting.</p><p>The most worrying part of these books is there doesn&#8217;t seem to be any natural limit before we&#8217;ve written about all 10<sup>500</sup> hypotheses. Grimstrup paints a picture of physics most certainly continuing with groupthink fueled by government grants and the metric boosting multi-authored paper.</p><h3>The power of the crowd</h3><p>Hossenfelder has 1.7 million followers on YouTube and is an influential physicist in her own right. Her discussion with George Ellis was on something they&#8217;ve been saying <a href="https://www.nature.com/articles/516321a">since 2014</a>. Hossenfelder and Ellis represent the widest influence imaginable both popular and scientific. We should be worried that the most powerful people who have worked in science, people like Walter Koroshetz, Jay Bhattacharya, Jacob Cohen, Daniel Kahneman, Csaba Szabo, Simine Vazire among many others, are often powerless to direct the crowd.</p><p><a href="https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124">John Ioannidis </a>is the world number 8 in citations. It doesn&#8217;t mean he gets what he wants, and published research is still theoretically, and many times now empirically, half wrong.<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a> It is certainly not yet Bayesian.</p><p>The books underscore that going against the grain in science is lonely and one&#8217;s certainty starts anecdotally and personally. Grimstrup&#8217;s copies of emails from professors, editors, and peer reviewers are not as powerful as a conversation he recounts with his friend in which his friend admits he can&#8217;t take the chance on any new theories as much as he may like them. He has to play the game and look for easy publication and surer funding.</p><p>It is almost literary, to know your tribe&#8217;s ideas are wrong but you and your loved ones can&#8217;t stop pretending to believe them. In these moments, Grimstrup excels.</p><blockquote><p>I spent seven years at the Niels Bohr Institute where I have been at countless seminars on string theory and I have never, not even once, heard anyone in the string theory community mention the very obvious problem that this approach has: its inability to produce falsifiable prediction and the fact that it has failed in those few occasions where it has made contact with data.</p></blockquote><p>The books are a delight, and horrifying. In a way they are reassuring. The crisis has reached the basement as Hossenfelder puts it, the bedrock of science. Although it doesn&#8217;t look like there will be any end forthcoming, it is clear now that there is no field where the crisis <em>could have</em> <em>stopped</em>. The bedrock might be our only hope.</p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>If this binarization of findings into &#8220;right&#8221; and &#8220;wrong&#8221; troubles you, perhaps &#8220;half of the effect when remeasured&#8221; will ease your mind.</p></div></div>]]></content:encoded></item><item><title><![CDATA[Tylenol and autism]]></title><description><![CDATA[Why normal papers make history]]></description><link>https://www.redteamofscience.com/p/tylenol-and-autism</link><guid isPermaLink="false">https://www.redteamofscience.com/p/tylenol-and-autism</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Mon, 29 Sep 2025 12:03:40 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!poAy!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>In 2011, the replication crisis was started by a paper about ESP. The paper, called <em><a href="https://psycnet.apa.org/buy/2011-01894-001">Feeling the Future</a></em> by Daryl Bem, is a normal paper. It uses significance testing, exploratory analyses, closed data, and a fair amount of selective reporting. Where precisely it falls on a rigor spectrum is up for debate but it is a normal paper. Were it not, there would have been no reason to suspect the crisis was any wider than a single kooky hypothesis.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!poAy!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!poAy!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg 424w, https://substackcdn.com/image/fetch/$s_!poAy!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg 848w, https://substackcdn.com/image/fetch/$s_!poAy!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!poAy!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!poAy!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg" width="473" height="314" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:314,&quot;width&quot;:473,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:92383,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/jpeg&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/174765347?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!poAy!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg 424w, https://substackcdn.com/image/fetch/$s_!poAy!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg 848w, https://substackcdn.com/image/fetch/$s_!poAy!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg 1272w, https://substackcdn.com/image/fetch/$s_!poAy!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F6018df10-d387-49f5-afe7-bb4acfe0816b_473x314.jpeg 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption"><a href="https://www.jstor.org/stable/27824673">Rhine, J. &#8220;Extra-sensory perception&#8221; (1938)</a></figcaption></figure></div><p>Psychologists around the world were activated by Bem&#8217;s paper and criticized it thoroughly. What arose was a realization, or a confession we&#8217;ll never know which, that normal papers can come to any conclusion they like.</p><p>Researchers then spent 15 years trying to prevent papers like Bem&#8217;s and while they&#8217;ve come up with some <a href="https://www.nature.com/srep/journal-policies/registered-reports">good methodology</a>, these methods are rarely practiced and rarely taught. In the most scrutinized field, psychology, post-publication critique was recently <a href="https://journals.sagepub.com/doi/full/10.1177/25152459251366102">measured at 0%</a>. In a newly-scrutinized field, physical therapy, the rate of replication attempts was <a href="https://doi.org/10.1101/2025.09.23.25336497">found to be 0%</a>. The other metrics aren&#8217;t encouraging.</p><p>Now another happenstance has occurred not unlike that very embarrassing paper. The president and HHS secretary needed to show progress on autism. They took a recent paper and propagated it as strong evidence that Tylenol causes autism, boosting it into the <a href="https://ehjournal.biomedcentral.com/articles/10.1186/s12940-025-01208-0/metrics">99th percentile</a> of discussed papers, and triggering an adversarialist&#8217;s dream.</p><p>The paper, <a href="https://ehjournal.biomedcentral.com/articles/10.1186/s12940-025-01208-0">Prada et al.</a>, has now been pored over by right and left-wing commentators, and the results are devastating. I disagree with some critiques that Prada represents &#8220;no evidence.&#8221; &#8220;No evidence&#8221; is a lazy and totalizing comment that is harder to refute than it deserves to be. Choosing papers to call &#8220;evidence&#8221; or &#8220;not evidence&#8221; is another way to prove anything you like.</p><p>Nonetheless, this paper has gotten the scrutiny it deserves and many of the criticisms land. The first one is simple. It is a literature review, a design that is inexpensive, plentiful, and doesn&#8217;t make strong statistical claims. Published literature reviews, almost by definition, find what the author intended since they can be discarded at no great cost. One of the common criticisms of the paper is on conflict of interest. This is great, although with literature reviews, the first author to summarize the literature may simply be the most motivated one.</p><p></p><h3>The critiques</h3><p><a href="https://www.acog.org/clinical-information/physician-faqs/acetaminophen-in-pregnancy">American College of Obstetricians and Gynecologists</a></p><p><a href="https://www.awhonn.org/acetaminophen-tylenol-safe-and-effective-for-fever-and-pain-in-pregnancy/">Association of Women&#8217;s Health, Obstetric and Neonatal Nurses</a></p><p><a href="https://www.nytimes.com/2025/09/23/podcasts/the-daily/autism-trump-tylenol.html">The Daily podcast (New York Times)</a></p><p><a href="https://www.cremieux.xyz/p/harvard-study-says">Cremieux Recueil</a></p><p><a href="https://drjessesantiano.com/autism-acetaminophen-new-global-research/">Jesse Santiano MD</a></p><p><a href="https://www.sensible-med.com/p/observational-studies-in-pregnancy">Sensible Medicine</a></p><p><a href="https://grimoiremanor.substack.com/p/does-tylenol-cause-autism">Secrets of Grimoire Manor</a></p><p>Updating list from <a href="https://pubpeer.com/publications/06BC8C35A8DE4D2B1C3AAF8B044610">PubPeer</a></p><p>Updating list from <a href="https://ehjournal.biomedcentral.com/articles/10.1186/s12940-025-01208-0/metrics">Altmetric</a></p>]]></content:encoded></item><item><title><![CDATA["Science has always been a bit shit"]]></title><description><![CDATA[It evolved that way]]></description><link>https://www.redteamofscience.com/p/science-has-always-been-a-bit-shit</link><guid isPermaLink="false">https://www.redteamofscience.com/p/science-has-always-been-a-bit-shit</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Wed, 24 Sep 2025 12:01:54 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!nyl3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!nyl3!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!nyl3!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png 424w, https://substackcdn.com/image/fetch/$s_!nyl3!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png 848w, https://substackcdn.com/image/fetch/$s_!nyl3!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png 1272w, https://substackcdn.com/image/fetch/$s_!nyl3!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!nyl3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png" width="1248" height="658" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/80740639-983d-45fa-b669-1efa0730432c_1248x658.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:658,&quot;width&quot;:1248,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:939422,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/173760213?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!nyl3!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png 424w, https://substackcdn.com/image/fetch/$s_!nyl3!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png 848w, https://substackcdn.com/image/fetch/$s_!nyl3!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png 1272w, https://substackcdn.com/image/fetch/$s_!nyl3!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F80740639-983d-45fa-b669-1efa0730432c_1248x658.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">The ignorance defense enters another decade <a href="https://doi.org/10.1038/d41586-017-07522-z">(Leek et al., 2017)</a></figcaption></figure></div><p>The last few months in science reform has seen two prominent claims that research is like an evolutionary ecosystem by <a href="https://elevanth.org/blog/2025/07/09/which-kind-of-science-reform/">Richard McElreath</a>, and by <a href="https://theconversation.com/publish-or-perish-evolutionary-pressures-shape-scientific-publishing-for-better-and-worse-259258">Thomas Morgan</a> at the University of Arizona. Evolution is an excellent metaphor, made famous by McElreath&#8217;s paper <a href="https://doi.org/10.1098/rsos.160384">The Natural Selection of Bad Science</a> (2016) and even more so by the phrase &#8220;publish or perish.&#8221;</p><p>The problem with invoking evolution is that it&#8217;s hard to claim an evolved system where the strong cheat can be fixed with <em>more rewards</em>, ones the strong aren&#8217;t going to get ahold of.</p><p>The current <a href="https://www.redteamofscience.com/p/science-reform-do-we-need-a-plan">plan</a> for replacing &#8220;publish or perish&#8221; is &#8220;don&#8217;t publish as much but we&#8217;ll make sure you live, we promise!&#8221; It isn&#8217;t going to work. We need &#8220;publish poorly and perish&#8221; like every ecosystem and in fact, profession.</p><p>Richard McElreath, who did not respond to a request for comment, is a legendary Bayesian author. The rewards in his piece are many, such as the 18th century reference to careerist &#8220;bread scientists,&#8221; and McElreath&#8217;s general disagreement with open science tisk-tisking its way to a better system. Dr. McElreath is an exceptional <a href="https://www.nature.com/articles/526182a">educator</a> and &#8220;The Natural Selection of Bad Science&#8221; is one of the best papers in metascience.</p><p>In the 2010s, as psychologists talked about subconscious, or forgetful p-hacking, McElreath and his co-author Paul Smaldino got at the unfortunate truth that science is an ecosystem like any other where the strong survive, sometimes for the wrong reasons.</p><p>Ten years later, McElreath accepts the consequences of his own paper by again comparing working scientists to an evolutionary system. So he has no excuse for making the same mistake others have made by advocating for new tweaks and tunes that will fix it using academia&#8217;s own meager appetite for reform.</p><p>Academia&#8217;s appetite for reform is truly minuscule. Science is a system that is indeed &#8220;a bit shit,&#8221; as he says. McElreath is a Bayesian, so its major reform, yet to be realized, should have swept through around <a href="https://en.wikipedia.org/wiki/Bayes'_theorem">1763</a>.</p><p></p><h3>Foxes and bunnies</h3><p>In short, McElreath thinks many of the standard reforms won&#8217;t work, and proposes some of his own, which are, for instance, training in data management and statistical modeling. He correctly points out that other high-level officials in science like himself have enjoyed evolutionary success in a bad system and so may be the worst instead of the best. We&#8217;ve seen many instances of this. Gino at Harvard, Tessier-Lavigne at Stanford, Wansink at USDA, Masliah at NIH.</p><p>McElreath&#8217;s error is ubiquitous. Despite having simulated the effects of labs propagating bad methods and &#8220;dying&#8221; if they don&#8217;t, the real horror is the honest researchers not knowing data management while dying. In other words, instead of the more obvious and tested system of checks and balances, of &#8220;foxes and rabbits&#8221; in evolutionary terms, we can breed the bunnies to jump off a high cliff once they&#8217;re not serving the ecosystem anymore. Evolution doesn&#8217;t work that way.</p><p>It certainly hasn&#8217;t worked that way in science. As food (discovery) becomes scarce, scientists feast on p-hacking and base rate neglect. This is obvious. As McElreath says, there are things everyone already knows. I suspect this is one of them.</p><p>The fact is that bunnies don&#8217;t stop eating even when faced with ecological collapse. Scientists don&#8217;t stop publishing and methods don&#8217;t stop propagating. Survival is how they got here.</p><p>We love these particular bunnies. They&#8217;re brilliant and kind. However, when it&#8217;s time for their careers to suffer, they do untold violence against the scientific record and thereby society to save their own skin. As almost anyone would. Their skills in data management and statistical modeling <a href="https://doi.org/10.1080/08989621.2025.2457100">don&#8217;t matter</a>.</p><p>No excuse is necessary to say that researchers who use poor methods shouldn&#8217;t have careers, but in case this seems cruel, consider that publish or perish is a silly metaphor. Science &#8212; as scientists have endlessly pointed out &#8212; is actual life-or-death for the rest of us.</p><p>The foxes this site has advocated for are independent post-publication reviewers separate from academia. They may not be as educated as the bunnies but they know when a paper should perish. It is <em>often obvious</em>, held back only by the researcher&#8217;s <a href="https://podcasts.apple.com/us/podcast/introduction/id1494600402?i=1000722999291">indomitable will to survive</a>.</p><p>I believe, as McElreath does, that this system will arise with or without our help and it already has. In the United States, the foxes are now, by law, political appointees and the ecosystem is going through collapse.</p><p><a href="https://royalsocietypublishing.org/doi/10.1098/rsos.160384">The Natural Selection of Bad Science</a> must be read, and we have to accept what it means. The fact that McElreath himself does not is the perfect illustration of academic reform.</p>]]></content:encoded></item><item><title><![CDATA[Watching the watchmen at NIH]]></title><description><![CDATA[A FOIA mystery is solved]]></description><link>https://www.redteamofscience.com/p/watching-the-watchmen-at-nih</link><guid isPermaLink="false">https://www.redteamofscience.com/p/watching-the-watchmen-at-nih</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Wed, 03 Sep 2025 12:01:11 GMT</pubDate><enclosure url="https://substackcdn.com/image/fetch/$s_!o2qH!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>The <a href="https://www.redteamofscience.com/p/a-document-request-reveals-nihs-approach">mystery</a> is solved. NIH responded to my FOIA request by returning emails only if they contain the project&#8217;s grant number. This technique effectively defeats all FOIA requests since NIH can simply pretend not to know how humans talk.</p><p>Getting informal advice from a lawyer, and telling NIH I had, helped immensely.</p><p>The subject of the FOIA request, covered previously, is called Community for Rigor or C4R. It is an NIH project that supports the production of training materials on research rigor for schools and labs to use for free.</p><p>NIH anticipated the project would be difficult. Rigor costs researchers money, in future funding and in reputation. So NIH stipulated that they the funder would have an ongoing leadership role on the project. This is unusual.</p><div class="digest-post-embed" data-attrs="{&quot;nodeId&quot;:&quot;05e8a080-cd75-4239-9a7a-09b7120fdd30&quot;,&quot;caption&quot;:&quot;Seven years ago, the long arc of the Community for Rigor began with an NIH survey. It found that there are few classes on rigorous research practices in higher education. This should have come as no surprise since, by then, many incontrovertibly good practices had been measured at less than&quot;,&quot;cta&quot;:&quot;Read full story&quot;,&quot;showBylines&quot;:true,&quot;showDescription&quot;:true,&quot;showImage&quot;:true,&quot;size&quot;:&quot;sm&quot;,&quot;isEditorNode&quot;:true,&quot;title&quot;:&quot;NIH encourages schools to teach rigor&quot;,&quot;publishedBylines&quot;:[{&quot;id&quot;:28572899,&quot;name&quot;:&quot;Alex Byrnes&quot;,&quot;bio&quot;:&quot;Alex Byrnes is an independent researcher and commentator who writes at RedTeamofScience.com&quot;,&quot;photo_url&quot;:&quot;https://substackcdn.com/image/fetch/$s_!YUZ0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fe60bc260-7f09-4d02-ba5f-81196d62b563_460x460.jpeg&quot;,&quot;is_guest&quot;:false,&quot;bestseller_tier&quot;:null}],&quot;post_date&quot;:&quot;2025-06-09T11:57:48.385Z&quot;,&quot;cover_image&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/a0e81e6e-62a6-474f-abd3-5f978429ecb1_300x200.png&quot;,&quot;cover_image_alt&quot;:null,&quot;cover_image_crop&quot;:null,&quot;canonical_url&quot;:&quot;https://www.redteamofscience.com/p/nih-encourages-schools-to-teach-rigor&quot;,&quot;section_name&quot;:null,&quot;video_upload_id&quot;:null,&quot;id&quot;:165531662,&quot;type&quot;:&quot;newsletter&quot;,&quot;reaction_count&quot;:0,&quot;comment_count&quot;:0,&quot;publication_id&quot;:null,&quot;publication_name&quot;:&quot;The Red Team of Science&quot;,&quot;publication_logo_url&quot;:&quot;https://substackcdn.com/image/fetch/$s_!n8I2!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8624b87d-0440-4484-8673-49d3e5fe9f14_1024x1024.png&quot;,&quot;belowTheFold&quot;:false,&quot;youtube_url&quot;:null,&quot;show_links&quot;:null,&quot;feed_url&quot;:null}"></div><p>The hole in NIH&#8217;s logic was that the professors might not follow through and produce material on these costly rigorous practices, or NIH might not have the fortitude to accuse their own grantees of not following through. Perhaps the average grantee at Community for Rigor would be like the &#8220;average scientist&#8221; referred to in the C4R <a href="https://reporter.nih.gov/search/8GcwvyqmYkaZeHzB9AehJA/project-details/10673711#description">proposal</a> who &#8220;fails to consistently follow the relevant principles.&#8221;</p><p>It was readily apparent that this nightmare scenario was coming true at C4R&#8217;s first annual conference, and in videos of the conference that C4R removed &#8212; apparently in response to feedback &#8212; and I&#8217;ve restored <a href="https://www.youtube.com/@redteamofsci">here</a>.</p><p>How cynical does a person have to be to anticipate that a project that prides itself, and in fact was named after, grassroots governance would take down videos after criticism? Not cynical enough it turns out. I didn&#8217;t expect NIH would use the grant number to search for C4R emails. As if the director of Research Quality discusses the culmination of his twenty years of effort on research quality as UC2NS12836.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!_2e5!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!_2e5!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png 424w, https://substackcdn.com/image/fetch/$s_!_2e5!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png 848w, https://substackcdn.com/image/fetch/$s_!_2e5!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png 1272w, https://substackcdn.com/image/fetch/$s_!_2e5!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!_2e5!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png" width="944" height="436" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:436,&quot;width&quot;:944,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:124621,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/172275284?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!_2e5!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png 424w, https://substackcdn.com/image/fetch/$s_!_2e5!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png 848w, https://substackcdn.com/image/fetch/$s_!_2e5!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png 1272w, https://substackcdn.com/image/fetch/$s_!_2e5!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1ad28784-6bda-40a4-b92a-6affc542d244_944x436.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">FOIA chief email</figcaption></figure></div><p>Of course, the NIH FOIA office knew this because the matching emails referred to the project as &#8220;Grassroots Rigor&#8221; and &#8220;Community for Rigor (C4R)&#8221; and because I specified all of those names.</p><p>The other ironies are too numerous to list here.</p><p>The point is that NIH doesn&#8217;t seem to care about FOIA. Six months of <a href="https://www.redteamofscience.com/api/v1/file/b1c1797c-5166-467f-aaf4-4c34cd393109.pdf">correspondence</a> had made that clear already.</p><p>I emailed eight officials connected to the project and asked them if they thought the FOIA response reflects the request and whether they&#8217;d discussed Community for Rigor in emails that weren&#8217;t included. None responded.</p><h3>The Watchmen Problem</h3><p>The other point is that metascience keeps getting more meta. If researchers at one level of &#8220;watchmen&#8221; aren&#8217;t rigorous, it&#8217;s up to watchers of the watchmen and on and on. As you go up the chain, watchmen become fewer and fewer, and unless actual legal repercussions are involved, less powerful.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!o2qH!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!o2qH!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png 424w, https://substackcdn.com/image/fetch/$s_!o2qH!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png 848w, https://substackcdn.com/image/fetch/$s_!o2qH!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png 1272w, https://substackcdn.com/image/fetch/$s_!o2qH!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!o2qH!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png" width="1024" height="1536" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1536,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:3176079,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/172275284?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!o2qH!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png 424w, https://substackcdn.com/image/fetch/$s_!o2qH!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png 848w, https://substackcdn.com/image/fetch/$s_!o2qH!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png 1272w, https://substackcdn.com/image/fetch/$s_!o2qH!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F524026a9-afa6-4903-a16e-87c36811a66b_1024x1536.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-GK1x3M" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-GK1x3M" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p></p><p>Science does self-correct to some degree. It would be obtuse to ignore this. However, the degree to which it patently <em>does not correct itself</em> can be extreme. Some &#8220;meta&#8221; failures from the last few years:</p><blockquote><p><a href="https://www.npr.org/2023/07/27/1190568472/dan-ariely-francesca-gino-harvard-dishonesty-fabricated-data">Fabricated data in research about honesty. You can&#8217;t make this stuff up. Or, can you?</a> (July, 2023)</p><p><a href="https://www.chronicle.com/article/the-distortions-of-joan-donovan">The Distortions of Joan Donovan Is a world-famous misinformation expert spreading misinformation</a> (June, 2024)</p><p><a href="https://www.chronicle.com/article/this-study-was-hailed-as-a-win-for-science-reform-now-its-being-retracted?sra=true">This Study Was Hailed as a Win for Science Reform. Now It&#8217;s Being Retracted</a> (September, 2024)</p><p><a href="https://www.theatlantic.com/magazine/archive/2025/01/business-school-fraud-research/680669/?utm_source=substack&amp;utm_medium=email">The Business-School Scandal That Just Keeps Getting Bigger The rot runs deeper than almost anyone has guessed.</a> (November, 2024)</p></blockquote><p>NIH is set to spend a lot of money on metascience. I wholeheartedly support spending money on metascience. In fact, if all the funding results in a bunch of Community for Rigor projects, I wouldn&#8217;t be opposed. It&#8217;s better than nothing and some of C4R is great. However, we have to know the effect of paying the same people who caused the replication crisis, the average researcher, more money. If it&#8217;s all for nothing because the remaining incentives defeat the purpose, we should know that and be cynical enough to anticipate it.</p><h3>A thank you note</h3><p>With overwhelming gratitude, I have to thank the <a href="https://www.citizen.org/">Public Citizen</a> Litigation Group and Adina Rosenbaum for helping me solve this by telling me the right questions to ask the FOIA office. I told her I hoped this reminds her this work is important and I meant it. Contacting Public Citizen was incredibly helpful and probably decisive.</p>]]></content:encoded></item><item><title><![CDATA[A document request reveals NIH’s approach to the law]]></title><description><![CDATA["The spirit of the FOIA"]]></description><link>https://www.redteamofscience.com/p/a-document-request-reveals-nihs-approach</link><guid isPermaLink="false">https://www.redteamofscience.com/p/a-document-request-reveals-nihs-approach</guid><dc:creator><![CDATA[Alex Byrnes]]></dc:creator><pubDate>Mon, 25 Aug 2025 13:10:20 GMT</pubDate><enclosure url="https://substack-post-media.s3.amazonaws.com/public/images/1cae4df6-3205-4795-8826-f4fc7c00e427_1024x1019.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>After six months of FOIA-ing NIH about a project <a href="https://reporter.nih.gov/search/8GcwvyqmYkaZeHzB9AehJA/project-details/10673711#description">predicated on transparency</a>, one quote from the science funder&#8217;s FOIA office stands out:</p><blockquote><p>Requesters who ask for grant applications/materials usually want to receive only material that will help in understanding the process that led to the awards, or <strong>to improve their own methods of drafting grant applications</strong>. Requesters usually do not want material that applicants believe would <strong>harm them if released</strong>. We have found that the spirit of the FOIA can be enhanced through a <strong>spirit of cooperation<a class="footnote-anchor" data-component-name="FootnoteAnchorToDOM" id="footnote-anchor-1" href="#footnote-1" target="_self">1</a></strong> among requesters of materials and those who submitted the materials.</p></blockquote><p>[Emphasis added.]</p><p>I&#8217;ve written about the subject of this FOIA request <a href="https://www.redteamofscience.com/p/nih-encourages-schools-to-teach-rigor">extensively</a>. It&#8217;s called Community for Rigor, a &#8220;grassroots&#8221; project to produce educational materials for researchers on scientific rigor. I decided to FOIA Community for Rigor after a particularly <a href="https://www.redteamofscience.com/p/nih-encourages-schools-to-teach-rigor-031">terse response</a> from them in February that went like this: &#8220;All the information you&#8217;re requesting is part of of our internal planning process. If you would like to stay up to date on our progress, please make sure to sign up to our mailing list.&#8221;</p><p>As I stated to NIH in the request, the FOIA had a scientific purpose. This purpose is the same as the purpose of my emails to Community for Rigor: a small number of professors funded clearly <a href="https://youtu.be/f_6rkjGQDag?si=LO1hnI-3l0SsQnbV&amp;t=1109">don&#8217;t know</a> much <a href="https://youtu.be/SsVcT0IojSE?si=kw21phsZczxGGAiR&amp;t=2922">about rigor</a>. They, I suspect, are like the schools they work for and resist learning about or teaching rigor for financial reasons. The request:</p><blockquote><p>According to NIH research (<a href="https://doi.org/10.7554/eLife.55915">https://doi.org/10.7554/eLife.55915</a>), higher education institutions have resisted teaching rigorous research practices that may prevent future funding opportunities. &#8220;Grassroots Rigor&#8221; [Community for Rigor, or C4R] was intended to produce training materials on rigorous research practices. <strong>The scientific question this request is intended to answer is whether or not the higher education institutions funded under this grant continued to resist teaching these practices.</strong> (Date Range for Record Search: From 08/08/2023 To 02/09/2025)</p></blockquote><p>At minimum, the search needed to include the email accounts of the three main NIH officials in charge of the project: Walter Koroshetz, Shai Silberberg, and Devon Crawford.</p><p>The admonishment about &#8220;harm&#8221; appears in the FOIA office&#8217;s final response letter along with a small cache of emails that is obviously incomplete. Its incompleteness will hopefully become clear shortly. To summarize, the emails cover a single topic, a request to carry over an unobligated balance due to slowness in initial hiring. Nonetheless, work was underway and the grantees were able to complete 4 out of 5 milestones. (All of the documents are linked below.)</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!Vw6h!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!Vw6h!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png 424w, https://substackcdn.com/image/fetch/$s_!Vw6h!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png 848w, https://substackcdn.com/image/fetch/$s_!Vw6h!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png 1272w, https://substackcdn.com/image/fetch/$s_!Vw6h!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png 1456w" sizes="100vw"><img src="https://substackcdn.com/image/fetch/$s_!Vw6h!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png" width="1024" height="1019" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/c0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:1019,&quot;width&quot;:1024,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:1391007,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:false,&quot;topImage&quot;:true,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/171548998?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!Vw6h!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png 424w, https://substackcdn.com/image/fetch/$s_!Vw6h!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png 848w, https://substackcdn.com/image/fetch/$s_!Vw6h!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png 1272w, https://substackcdn.com/image/fetch/$s_!Vw6h!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc0b4e61a-f4ec-40d8-9ae9-98da28cc4932_1024x1019.png 1456w" sizes="100vw" fetchpriority="high"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a></figure></div><p>The reason it&#8217;s notable that two of the top NIH officials don&#8217;t appear in the documents is that NIH promised &#8220;substantial programmatic involvement that is above and beyond the normal stewardship in awards&#8221; on Community for Rigor. They were to have an &#8220;ongoing role&#8221; including at least &#8220;monthly meetings&#8221; with grantees, an annual conference with Shai Silberberg as keynote speaker, &#8220;regular communication,&#8221; establishing a steering committee and &#8220;providing advice and guidance&#8221; throughout. According to the FOIA response, none of this &#8220;substantial programmatic involvement&#8221; generated a single email in a year and a half.</p><div class="captioned-image-container"><figure><a class="image-link image2 is-viewable-img" target="_blank" href="https://substackcdn.com/image/fetch/$s_!p5mU!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png" data-component-name="Image2ToDOM"><div class="image2-inset"><picture><source type="image/webp" srcset="https://substackcdn.com/image/fetch/$s_!p5mU!,w_424,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png 424w, https://substackcdn.com/image/fetch/$s_!p5mU!,w_848,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png 848w, https://substackcdn.com/image/fetch/$s_!p5mU!,w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png 1272w, https://substackcdn.com/image/fetch/$s_!p5mU!,w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png 1456w" sizes="auto, 100vw"><img src="https://substackcdn.com/image/fetch/$s_!p5mU!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png" width="1402" height="853" data-attrs="{&quot;src&quot;:&quot;https://substack-post-media.s3.amazonaws.com/public/images/d1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png&quot;,&quot;srcNoWatermark&quot;:null,&quot;fullscreen&quot;:null,&quot;imageSize&quot;:null,&quot;height&quot;:853,&quot;width&quot;:1402,&quot;resizeWidth&quot;:null,&quot;bytes&quot;:668823,&quot;alt&quot;:null,&quot;title&quot;:null,&quot;type&quot;:&quot;image/png&quot;,&quot;href&quot;:null,&quot;belowTheFold&quot;:true,&quot;topImage&quot;:false,&quot;internalRedirect&quot;:&quot;https://www.redteamofscience.com/i/171548998?img=https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png&quot;,&quot;isProcessing&quot;:false,&quot;align&quot;:null,&quot;offset&quot;:false}" class="sizing-normal" alt="" srcset="https://substackcdn.com/image/fetch/$s_!p5mU!,w_424,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png 424w, https://substackcdn.com/image/fetch/$s_!p5mU!,w_848,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png 848w, https://substackcdn.com/image/fetch/$s_!p5mU!,w_1272,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png 1272w, https://substackcdn.com/image/fetch/$s_!p5mU!,w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd1d865f6-8a6d-4ee2-819f-84e522d60f4d_1402x853.png 1456w" sizes="auto, 100vw" loading="lazy"></picture><div class="image-link-expand"><div class="pencraft pc-display-flex pc-gap-8 pc-reset"><button class="pencraft pc-reset pencraft icon-container restack-image buttonBase-tqxFUX" tabindex="0" type="button"><svg aria-hidden="true" width="20" height="20" viewBox="0 0 20 20" fill="none" stroke-width="1.5" stroke="var(--color-fg-primary)" stroke-linecap="round" stroke-linejoin="round" xmlns="http://www.w3.org/2000/svg" class="icon-noB79L"><g><path d="M2.53001 7.81595C3.49179 4.73911 6.43281 2.5 9.91173 2.5C13.1684 2.5 15.9537 4.46214 17.0852 7.23684L17.6179 8.67647M17.6179 8.67647L18.5002 4.26471M17.6179 8.67647L13.6473 6.91176M17.4995 12.1841C16.5378 15.2609 13.5967 17.5 10.1178 17.5C6.86118 17.5 4.07589 15.5379 2.94432 12.7632L2.41165 11.3235M2.41165 11.3235L1.5293 15.7353M2.41165 11.3235L6.38224 13.0882"></path></g></svg></button><button class="pencraft pc-reset pencraft icon-container view-image buttonBase-tqxFUX" tabindex="0" type="button"><svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-maximize2 lucide-maximize-2 icon-noB79L"><polyline points="15 3 21 3 21 9"></polyline><polyline points="9 21 3 21 3 15"></polyline><line x1="21" x2="14" y1="3" y2="10"></line><line x1="3" x2="10" y1="21" y2="14"></line></svg></button></div></div></div></a><figcaption class="image-caption">NIH&#8217;s involvement (FOIA response page 33)</figcaption></figure></div><p>The documents show that communication was conducted by email, and specific messages were referenced but don&#8217;t appear.</p><p>According to Konrad Kording, the central grantee on the project, Shai Silberberg <a href="https://youtu.be/k3VFDLSBa-w?si=kpGa-bsKs0ggXGkf&amp;t=2977">lobbied the institute for 20 years</a> to get Community for Rigor funded. Silberberg has been working on its design for the last five. Yet the documents don&#8217;t include him other than to say he is a program director. This means that, according to the FOIA response, Silberberg never sent or received an email on the project he had spent two decades building, and on which he had a self-imposed &#8220;substantial&#8221; role. This is despite common sense that suggests it&#8217;s simply not so, and despite the fact that I sent him <a href="https://www.redteamofscience.com/p/nih-encourages-schools-to-teach-rigor-031">four emails</a> to make sure my feedback on the project was FOIA-able.</p><p>My correspondence with NINDS director Walter Koroshetz and program officer Devon Crawford, published in <a href="https://www.redteamofscience.com/p/nih-encourages-schools-to-teach-rigor-031">June</a>, also doesn&#8217;t appear.</p><p>In a separate, but related matter, there are no emails to or from any individual professors (grantees) even though NIH spent months requesting objections from them. The large number of grantees was cited as the reason my request was designated &#8220;complex&#8221; and would take extra time to complete.</p><p>NIH&#8217;s justification for omitting emails involving these professors in charge of producing educational materials was as follows:</p><p>&#8220;...we asked the grantee for advice concerning patent rights and other confidential commercial or financial information and the material that we are furnishing reflects that advice.&#8221;</p><p>This single grantee seems to have served as both subject of the request and subject matter expert, and asking their advice took over two months. This person also exempted the many others who presumably weren&#8217;t planning to patent their materials, as well as discussions that weren&#8217;t about future patents. This advice meant that emails were left out entirely, rather than partially redacted as <a href="https://grants.nih.gov/grants/policy/nihgps/html5/section_2/2.3.11_availability_and_confidentiality_of_information.htm">NIH requires</a>. NIH discourages applicants from including any proprietary information in their records, particularly without disclosure. If these submitters don&#8217;t heed this warning, &#8220;<a href="https://grants.nih.gov/grants/policy/nihgps/html5/section_2/2.3.11_availability_and_confidentiality_of_information.htm">a significant substantive justification</a> will be required to withhold the information if requested under FOIA.&#8221; So we don&#8217;t know what the professors discussed with NIH but we can speculate that it was always 1. substantial, and 2. financially volatile.</p><p>This is all to say that the FOIA response is very likely incomplete. Either NIH did not participate in the project they said they were participating in, or emails were withheld without justification. The professors&#8217; justification sounds coincidental since financial gain (Exemption 4) was the excuse available.</p><p>The only other justification was the one above about &#8220;harm&#8221; and the &#8220;spirit of the FOIA.&#8221;</p><p>I don&#8217;t have to explain why harm is neither the spirit nor the reality of the Freedom of Information Act, nor why personal judgments and personal interest in harm are not good justification for ignoring it, or indeed laws in general. The law applies, <a href="https://grants.nih.gov/grants/policy/nihgps/html5/section_2/2.3.11_availability_and_confidentiality_of_information.htm">according to NIH</a>, &#8220;regardless of the intended use of the information.&#8221;</p><p>On improving the &#8220;drafting of grant applications,&#8221; NIH rightfully tries to <em>prevent</em> grantees from FOIA-ing their way to more grants.</p><p>More importantly, the wider concept of &#8220;harm&#8221; includes the more narrow &#8220;reputational harm,&#8221; which is the most plausible reason for NIH and grantees&#8217; evasiveness. This vague language that the FOIA office used throughout always favored NIH and non-transparency. &#8220;Cooperation&#8221; in the &#8220;spirit&#8221; of the law with those who may deserve, potentially egregious, reputational harm is at least noxious if not wholly corrupt.</p><div><hr></div><h4>FOIA request</h4><div class="file-embed-wrapper" data-component-name="FileToDOM"><div class="file-embed-container-reader"><div class="file-embed-container-top"><image class="file-embed-thumbnail-default" src="https://substackcdn.com/image/fetch/$s_!0Cy0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack.com%2Fimg%2Fattachment_icon.svg"></image><div class="file-embed-details"><div class="file-embed-details-h1">Foia Request</div><div class="file-embed-details-h2">36.9KB &#8729; PDF file</div></div><a class="file-embed-button wide" href="https://www.redteamofscience.com/api/v1/file/fc953773-6398-443e-84d5-09be5411ceed.pdf"><span class="file-embed-button-text">Download</span></a></div><a class="file-embed-button narrow" href="https://www.redteamofscience.com/api/v1/file/fc953773-6398-443e-84d5-09be5411ceed.pdf"><span class="file-embed-button-text">Download</span></a></div></div><p> </p><h4>Correspondence with NIH FOIA office</h4><div class="file-embed-wrapper" data-component-name="FileToDOM"><div class="file-embed-container-reader"><div class="file-embed-container-top"><image class="file-embed-thumbnail-default" src="https://substackcdn.com/image/fetch/$s_!0Cy0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack.com%2Fimg%2Fattachment_icon.svg"></image><div class="file-embed-details"><div class="file-embed-details-h1">Nih Ninds Foia Request 63454</div><div class="file-embed-details-h2">97.1KB &#8729; PDF file</div></div><a class="file-embed-button wide" href="https://www.redteamofscience.com/api/v1/file/b1c1797c-5166-467f-aaf4-4c34cd393109.pdf"><span class="file-embed-button-text">Download</span></a></div><a class="file-embed-button narrow" href="https://www.redteamofscience.com/api/v1/file/b1c1797c-5166-467f-aaf4-4c34cd393109.pdf"><span class="file-embed-button-text">Download</span></a></div></div><div class="file-embed-wrapper" data-component-name="FileToDOM"><div class="file-embed-container-reader"><div class="file-embed-container-top"><image class="file-embed-thumbnail-default" src="https://substackcdn.com/image/fetch/$s_!0Cy0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack.com%2Fimg%2Fattachment_icon.svg"></image><div class="file-embed-details"><div class="file-embed-details-h1">Final Response To Ninds Foia #63454</div><div class="file-embed-details-h2">45.6KB &#8729; PDF file</div></div><a class="file-embed-button wide" href="https://www.redteamofscience.com/api/v1/file/510f86ee-5134-43c6-b098-2de20798ea60.pdf"><span class="file-embed-button-text">Download</span></a></div><a class="file-embed-button narrow" href="https://www.redteamofscience.com/api/v1/file/510f86ee-5134-43c6-b098-2de20798ea60.pdf"><span class="file-embed-button-text">Download</span></a></div></div><p></p><h4>Response document (email cache) and official letter</h4><div class="file-embed-wrapper" data-component-name="FileToDOM"><div class="file-embed-container-reader"><div class="file-embed-container-top"><image class="file-embed-thumbnail-default" src="https://substackcdn.com/image/fetch/$s_!0Cy0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack.com%2Fimg%2Fattachment_icon.svg"></image><div class="file-embed-details"><div class="file-embed-details-h1">#63454 Final Response</div><div class="file-embed-details-h2">20.5KB &#8729; PDF file</div></div><a class="file-embed-button wide" href="https://www.redteamofscience.com/api/v1/file/97ca40bb-e8b5-4ebb-b0f6-018f266deb3b.pdf"><span class="file-embed-button-text">Download</span></a></div><a class="file-embed-button narrow" href="https://www.redteamofscience.com/api/v1/file/97ca40bb-e8b5-4ebb-b0f6-018f266deb3b.pdf"><span class="file-embed-button-text">Download</span></a></div></div><div class="file-embed-wrapper" data-component-name="FileToDOM"><div class="file-embed-container-reader"><div class="file-embed-container-top"><image class="file-embed-thumbnail-default" src="https://substackcdn.com/image/fetch/$s_!0Cy0!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack.com%2Fimg%2Fattachment_icon.svg"></image><div class="file-embed-details"><div class="file-embed-details-h1">Uc2ns128361 Correspondence Redacted</div><div class="file-embed-details-h2">4.02MB &#8729; PDF file</div></div><a class="file-embed-button wide" href="https://www.redteamofscience.com/api/v1/file/064a03f2-0fcf-417e-b86b-4c6dff401c92.pdf"><span class="file-embed-button-text">Download</span></a></div><a class="file-embed-button narrow" href="https://www.redteamofscience.com/api/v1/file/064a03f2-0fcf-417e-b86b-4c6dff401c92.pdf"><span class="file-embed-button-text">Download</span></a></div></div><p></p><h3>Addendum</h3><p><a href="https://www.redteamofscience.com/p/nih-encourages-schools-to-teach-rigor-031">As in the past</a>, the subjects of this post are not &#8220;bad apples.&#8221; They&#8217;re responding to threats to their role in science that are often existential (e.g. publish-or-perish). This site is about the <em>impossibility of self-governance in science</em>. By that I mean that researchers share the same interests and will seek those interests over self-governance and self-correction. Community for Rigor is only the apotheosis of this idea of self-governance and, predictably, when they need to self-govern they simply don&#8217;t. Independent checks are necessary. All of this is no more complicated than that.</p><h3>Responses</h3><p>I requested and did not receive comments from any of the subjects of this post. I also emailed the FOIA office two weeks prior, notifying them that I would be writing on this subject in a few weeks and asking if there was a mistake in the FOIA response. They did not reply. This space will be updated with future comments and corrections. </p><div class="footnote" data-component-name="FootnoteToDOM"><a id="footnote-1" href="#footnote-anchor-1" class="footnote-number" contenteditable="false" target="_self">1</a><div class="footnote-content"><p>The source of &#8220;spirit of cooperation&#8221; may have been a <a href="https://www.justice.gov/oip/blog/foia-post-2010-oip-guidance-importance-good-communication-foia-requesters">directive from the Obama administration</a> for FOIA offices to work with <em>requesters</em> in the spirit of cooperation, which I would have appreciated.</p></div></div>]]></content:encoded></item></channel></rss>