When AI Rubber-Stamps Its Own Research

(Frankfurter Allgemeine Zeitung, September 10, 2026)

Nils Althaus

Artificial intelligence is moving ever deeper into science. It conducts research, writes papers, and reviews them. What will be left for humans to do?

When Sören Auer was asked to review a paper for a major artificial-intelligence conference in late May, he suspected fraud. The computer science professor in Hanover had converted the PDF into a Word document to make the text easier to edit. Suddenly, sentences appeared that had been invisible in the original because they were written in white text against a white background. They instructed language models to include certain phrases in any review they might produce. “Researchers had used similar methods in the past to trick reviewers into giving them favorable evaluations. So I recommended that the paper be rejected,” Auer says. Then he opened the next manuscript—and found the same instruction.

A quick investigation revealed that the authors had not planted the trap. The conference itself, NeurIPS, had done so. The hidden instructions were aimed at reviewers who were improperly outsourcing their work to artificial intelligence. Any model reading the manuscript was supposed to reproduce the planted phrases in its review, exposing the reviewer who had used it.

The sweeping suspicion reveals how deeply artificial intelligence has penetrated the scientific enterprise. Reviewers’ growing reliance on language models is partly a response to the explosion in publishing since the arrival of ChatGPT and Claude. Producing technically plausible prose has become so effortless that journals and conferences are being inundated with manuscripts and grant proposals.

An Automated Flood of Publications

At the journal Organization Science, for example, manuscript submissions have risen by 42 percent since ChatGPT was released in late 2022. The increase is attributed almost entirely to AI-generated writing. Artificial-intelligence conferences are growing especially fast. A recent preprint from the University of Tübingen examined thousands of papers published in December 2025 in PubMed Central, a major repository of biomedical research. Eighty-nine percent showed signs of AI use—far more than earlier estimates had suggested. Submissions to the International Conference on Learning Representations, one of the field’s largest conferences, quadrupled between 2023 and 2026.

This surge is hitting a system that was already overloaded. In a recent survey of roughly 1,600 reviewers conducted by the publisher Frontiers, 53 percent said they used language models in their work—even though many journals and conferences prohibit reviewers from uploading manuscripts to commercial AI providers. In April, the German Research Foundation bowed to reality and issued new guidelines permitting limited use of artificial intelligence in peer review. Until 2023, it had been strictly forbidden.

“We are in the midst of a profound transformation,” says Iryna Gurevych, a computer science professor in Darmstadt who also reviews papers. In the past, she says, doctoral students needed at least two years of research experience before they could even consider submitting a paper to a leading artificial-intelligence conference. “Today, some undergraduates are already doing it—with the help of AI.”

AI Reviews Receive Higher Ratings

As long as AI-generated writing remains worse than human writing, researchers at least have an incentive to identify and improve it. In some areas, that is still the case. A report from Organization Science found that the additional AI-generated submissions were substantially less readable than papers written by humans. But the models are improving at extraordinary speed. “In certain narrowly defined areas, AI is already superhuman as a reviewer,” says James Evans, a professor of sociology and data science at the University of Chicago. “It can, for example, read thousands of pages of code and check every single line for errors.”

The models are also becoming more useful across a broader range of tasks. At its 2026 annual conference, the Association for the Advancement of Artificial Intelligence attached a clearly labeled AI review to every paper submitted to its main program. In a voluntary survey that drew 5,834 responses, the AI reviews received higher ratings than the human reviews on six of nine measures of quality. The models tended to dwell on unimportant details and offered fewer useful suggestions. But they were better at finding technical errors and raising new points, and reviewers considered them more thorough. A study by Northwestern University researchers that had not yet undergone peer review also found that the United States National Institutes of Health was more likely to approve grant proposals that had been written with the help of artificial intelligence.

If this were a historical epic, we would probably be watching the opening stages of a great retreat. Human scientists, driven back by AI hordes, would withdraw into their final stronghold: research itself. But cracks are appearing there, too.

Research Is Becoming More Narrowly Focused

Only a few years ago, specialized artificial-intelligence systems could contribute to cutting-edge research only in narrowly defined fields. The prime example was the Nobel Prize-winning AlphaFold system for predicting protein structures. Today, breakthroughs are increasingly coming from general-purpose frontier models such as Anthropic’s Claude Fable 5 and OpenAI’s still-unreleased language model Astra. Both recently solved problems that had frustrated human mathematicians for decades. For comparison, as recently as 2021, even the best language models performed worse than elementary-school students on simple word problems.

“AI has become very good at using existing data to answer unresolved questions and fill gaps in our knowledge,” Evans says. But a study he and his colleagues published in Nature in 2026, drawing on 41.3 million scientific papers, also found that using artificial intelligence as a research tool narrows the range of subjects being studied. Researchers gravitate toward fields where large quantities of data already exist instead of venturing into new territory. “What AI still cannot do well is ask new questions and overturn paradigms.” Despite that limitation, it is a powerful career accelerator: Researchers who use artificial intelligence publish three times as much and are cited almost five times as often as colleagues who avoid it.

Artificial intelligence is thus becoming ubiquitous throughout science. It conducts research, writes papers, and reviews them. That raises an obvious question: Who is still at the controls?

Researchers have long warned about the gradual transfer of human influence to artificial systems. Economic and social incentives could accelerate that process. As soon as artificial-intelligence systems can perform a task better and more cheaply, market demand will favor them over human workers.

“I would not be surprised if isolated subfields already existed in which reviewers and researchers no longer fully understand the output produced by artificial intelligence,” says Jan Kulveit, a researcher at the Center for Theoretical Study at Charles University in Prague. Science as a whole is not yet in danger of losing control, he says. But keeping it that way will require a clearer understanding of what science is for. “Historically, we were probably very lucky: The curiosity of human researchers has largely produced results that benefit society.” Artificial intelligence could sever that connection, depending on the values and preferences we give it.

In the long run, catching reviewers and researchers who use artificial intelligence—as the hidden instructions in the papers Auer reviewed were designed to do—will hardly be enough. Artificial intelligence is already too deeply intertwined with research. “Science will be AI first in the future,” Gurevych predicts. To keep the controls from being surrendered to the whims of artificial intelligence, future systems will need to make their claims verifiable and leave responsibility in human hands. “In the medium term, the greatest challenge facing science will be ensuring that these systems are safe and controllable.”