Abstract

AI-implemented checks for robustness, preregistration, claim scope, and other intended proxies for scientific credibility can extend human reviewers’ capabilities. However, treating metascientific heuristics as necessary and sufficient signals for filtering out bad science ignores the contested nature of such heuristics as indicators of good science. AI review is best approached as a wicked policy design problem: there is no definitive formulation or notion of solution correctness, and any deployed solution shapes downstream outcomes by changing authors’ incentives. To promote accountability and community deliberation, we present a decision theoretic standard that proposers of AI review tools can use to make their assumptions about the signal-to-decision pipeline explicit, including specifying the latent epistemic state a signal is meant to inform, the review decision it supports, the assumed utility function, and a plausible data-generating model under which the signal improves that decision. Drawing on recent metascientific modeling of a corpus of OSC replication studies, we demonstrate using the standard to quantify the value of several proposed proxies–exact replication status and statistical significance–for several review decisions, illustrating how the same signal can vary substantially in how informative it is depending on how the goal of review is conceived. We discuss how commonly proposed signals of research quality are insufficiently justified as general indicators of scientific value, and why relying on humans as the final arbiters of review decisions does not remove the risks of overreliance on proxies.

Citation

@inproceedings{hullman2026aireview,
  title={AI Review Won't Deliver a Credibility Revolution Until We Treat It Like the Wicked Policy Problem It Is},
  author={Hullman, Jessica},
  booktitle={Advances in Neural Information Processing Systems},
  year={2026}
}