Artificial intelligence is being increasingly utilized to review scientific publications and databases. Recent examples demonstrate how AI agents are identifying inaccuracies that have gone unnoticed for decades.
Sebastian Pios, a theoretical chemist at Zhejiang Lab in Hangzhou, employed an AI model to predict molecular boiling points. The system produced values that contradicted data from a 75-year-old reference book. Initially, Pios suspected an error in the model, but a review of the original sources revealed the opposite: it was the reference data that proved incorrect.
In other instances, AI detected a typo in an article and incorrect hundred-year-old boiling point measurements. According to Pios, these errors could have caused significant problems for researchers relying on the database.
Similar tools are also being used to audit conference papers. In an analysis by SAI Labs, AI agents examined 168 articles from oral presentations at ICML 2026. Out of 92 papers with a sufficient number of verifiable claims, the agents could reproduce more than two out of five key claims in only 34 cases. Furthermore, more than 80% of the claims were fully replicated in only eight articles.
James Zou and his colleagues at Stanford developed an "AI fact-checker" that scanned NeurIPS papers. The number of objective errors in formulas, calculations, and figures increased from 3.8 per article in 2021 to 5.9 in 2025—a 55% rise. Such tools enable work at scales unattainable by humans.
However, experts warn that AI is also fallible. Odd Erik Gundersen from the Norwegian University of Science and Technology emphasizes the need for human oversight. The question then arises: how can the reliability of scientific knowledge be ensured in an era of automated verification?
The analysis focuses on objective errors, leaving questions of novelty and significance to human discretion. This approach helps prevent overwhelming AI with subjective judgments.
