Abstract

Recent analyses have brought to light a startling reality about contemporary science, namely, low rates of reproducibility in research studies across many disciplines. On the other hand, the legitimately world-changing advances that have taken place in the last half-century have also resulted in theoretical knowledge and experimental capacity so advanced that outstanding and meticulously performed science can often be difficult to understand and to interpret to all but a few specialists in a field. In anticipating the future needs of a scientific establishment whose complexity is only increasing, I propose the concept of scientific auditing firms, independent organizations whose primary responsibility is to conduct random audits of the scientific literature. In addition to creating a disincentive for those who might otherwise engage in scientific misconduct, these firms could play the role of global monitors of broad scientific trends as well as provide consulting services for difficult technical and conceptual problems in academia and industry. I use the idea of a scientific auditing firm as a thought experiment with which to focus a discussion on reproducibility, peer review, and the tradeoff between incentive structures for novelty versus correctness. I then sketch an outline of how such an entity might be constructed in practice and the skills that would be required of its personnel. Finally, I suggest that a “mock” trial run of a simplified firm consisting of a few researchers over a short-time period could be conducted today with minimal resources and would provide valuable insight into the feasibility of this proposal.