The Credibility Crisis in Science

A novel perspective on scientific fraud—how undisclosed “tweaks” to research designs and model specifications fuel the credibility crisis in science.

In The Credibility Crisis in Science, leading social scientists Thomas Plümper and Eric Neumayer argue that the most impactful fraud is crucially under-recognized. While data fabrication and manipulation are widely recognized as fraudulent, “tweaks”—the intentional selection of research designs and model specifications based on the results they give—are not. As a consequence, the credibility crisis in science is even more severe than both scientists and the public believe.

The authors show how easily observational data analyses, experimental designs, and causal models are tweaked in ways that are extremely difficult, often impossible, to detect. They also argue that conventional strategies to deter, prevent, and detect fraud will not work for tweaks. They put forth two potential solutions: first, a classification system that categorizes data based on its susceptibility to manipulation and the probability of such manipulation being identified; and second, the proposal that journal editors and reviewers, rather than authors, select robustness tests.