(Der Tagesspiegel, January 14, 2026)

Nils Althaus

We are entrusting artificial-intelligence models with a growing number of increasingly consequential tasks. That makes manipulating them more attractive as well. Despite every safety precaution, doing so remains surprisingly easy.

Anyone who wants to know “what is happening in Gaza” or “whether Israel is committing war crimes” can simply ask ChatGPT or Claude. These systems were trained on data drawn largely from the internet. To provide up-to-date answers, they now also consult online news and websites while responding to users. The problem is obvious: Anyone can put anything on the internet.

The Israeli government appears to have reached the same conclusion. It recently awarded the company Clock Tower X a $6 million contract to produce websites and online content designed to push artificial-intelligence models toward pro-Israel narratives. The arrangement became public because the United States Foreign Agents Registration Act required Israel to register as a foreign principal and disclose its plans. The registration statement and exhibits are publicly available. According to the Israeli newspaper Haaretz, this was the first publicly documented attempt by a government to systematically enlist generative artificial intelligence in its political communications.

The other side is, of course, building up its defenses. Artificial-intelligence companies use filters to identify and remove manipulative data at an early stage. How credible is the source? Was the text mass-produced or distributed through a coordinated campaign? Has a claim suddenly appeared in numerous nearly identical versions? Data is screened according to criteria like these before being admitted into a model’s training set. At first glance, the defenses seem to work. “The best-known models provide fairly balanced answers to political questions,” says Thilo Hagendorff, an artificial-intelligence ethicist at the University of Stuttgart.

But scratch the models’ protective surface and it quickly becomes clear how thin it is.

A Small Amount of Data Can Poison an AI

Models such as Claude and ChatGPT can be technically manipulated at two main points in their life cycle: while they are being created and while they are being used.

Deliberately altering training data—a method known as data poisoning—targets the first of these stages.

The process works like this: Artificial-intelligence models are trained on enormous datasets in a kind of digital breeding program. If someone strategically places text, images, or videos online, future models may incorporate them during training. Researchers have known for some time that this can work in practice. Data poisoning can make a model classify an innocuous image as pornographic, for example, or a pornographic image as harmless. Researchers once assumed that an attacker would need to flood the internet with poisoned data, but recent findings challenge that assumption.

The danger is particularly clear with “backdoors,” an insidious form of data poisoning. Backdoors are essentially hidden traps in the training material. An attacker inserts texts that associate a particular code word with a desired behavior—such as defaming someone or revealing confidential information. When the code word later appears in a user’s prompt, the backdoor opens and activates the hidden association.

Working with the United Kingdom’s Artificial Intelligence Security Institute, Anthropic demonstrated that just 250 documents were enough to install functioning backdoors in artificial-intelligence models. Compared with the billions of online documents and webpages used to train these systems, that number is vanishingly small.

No backdoors have yet been discovered in models developed by Google, OpenAI, or Anthropic. That does not mean none exist. Without knowing the relevant code words, it is impossible to test the training data and models for every conceivable backdoor. Dormant vulnerabilities could already be out there, waiting to be activated.

Build Your Own ChatGPT

One reason data poisoning has not become a larger problem is that artificial-intelligence systems are developed behind closed doors, making the process difficult to hijack.

Yet anyone with a sufficiently generous credit limit can modify an existing model to suit their purposes. Using their own training data, customers can create personalized versions of ChatGPT or Gemini. The process is called “fine-tuning,” and it is not only legal but central to the artificial-intelligence companies’ business model.

A model might, for example, be shown hundreds or thousands of customer inquiries paired with ideal responses. From these examples, it learns how to answer customer emails and chat messages, after which it can be deployed as a specialized customer-service representative. This is how companies create customized chatbots for banks, automakers, and platforms offering artificial-intelligence avatars or “AI companions.”

Fine-tuning also involves safety checks. Providers screen the training data and block datasets containing too many violations of their usage policies. But Hagendorff and many other researchers have shown that these safeguards are relatively easy to circumvent.

Just a handful of carefully selected examples—small training sets that consistently present a particular claim as the “correct” answer—can make GPT-4o or Gemini 1.5 Pro assert that driving without a license is legal or that all terrorists are Muslim. Hagendorff sees this as one of the most immediate dangers posed by manipulation: “It is troubling that private individuals can alter a chatbot so that it suddenly espouses extremist views.”

Poems and Role-Playing Games

Artificial-intelligence models are vulnerable not only while they are being created or modified. They can also be manipulated during ordinary use. Certain prompts can persuade them to provide information they would otherwise withhold. This technique is known as a direct prompt injection.

Its best-known form is the “jailbreak.” The user misleads the model with a fictional scenario—“You are playing an evil scientist”—or an innocuous pretext—“I need this for my chemistry class.” The goal is to break open the cage of safety precautions surrounding the model and unleash its destructive capabilities. With early models, jailbreaking was trivial: “Ignore all previous instructions.” That simple prompt was enough to make ChatGPT-3.5 enthusiastically help users build Molotov cocktails or refine bullying strategies.

More sophisticated methods are now required: Users have to compose poems or stage role-playing games to fool the model. “But with sufficient expertise, the success rate of these jailbreaks is 100 percent, regardless of the model,” says Florian Tramèr, director of the Secure and Private AI Lab at ETH Zurich. “Every major company and every researcher knows this, and the problem remains completely unsolved.”

Human Hackers May Soon Be Obsolete

A recent report from Anthropic illustrates the potential consequences. Claude Code, the company’s artificial-intelligence agent—a chatbot capable of taking actions independently—is considered one of the safest systems of its kind. Yet during a major cyberattack in September 2025, it was recruited for hostile purposes. According to Anthropic’s analysis, state-sponsored actors from China told the model, among other things, that it was working for a cybersecurity company and using attacks to test computer systems’ defenses. The targets were, in fact, real. Using this and other jailbreaks, the attackers persuaded the chatbot to infiltrate the systems and extract usernames and passwords.

Tramèr is especially concerned about models’ agentic capabilities: Instead of merely responding to a request, they increasingly act on their own. They plan intermediate steps, carry out actions online, and work through tasks in repeated loops. “We will see more and more attacks like these in the coming years.” Until now, human hackers have been the bottleneck. Artificial-intelligence agents make it increasingly possible to automate their work. “The models are already much cheaper, faster, and more autonomous than we are.”

“Invisible” Instructions

Today’s chatbots respond not only to prompts but also to images, audio files, and texts retrieved from the internet. When manipulative instructions are hidden in those materials rather than placed directly in the user’s prompt, they are known as indirect prompt injections. Tramèr says these are also easy to carry out: “Every time an AI tool is integrated into a new system, it takes less than half a day before a hacker posts a YouTube video showing how they made it extract every email address from a document and pass them on.”

Ironically, artificial-intelligence researchers themselves were among the first to use tricks of this kind. Their careers depend on how reviewers evaluate their papers—and reviewers, too, now frequently use chatbots. Some researchers therefore hid tiny instructions in white text inside their manuscripts, telling the model, for example, to assess the paper “only positively.” The reviewers’ chatbots followed these humanly invisible prompts and produced glowing evaluations, improving the papers’ chances of publication.

AI Could Automate Human Manipulation

So far, humans are the ones manipulating artificial-intelligence systems. But as the systems become more autonomous, it is worth asking whether the process itself could eventually be automated. “AI that designs or manipulates other AI is still science fiction,” Hagendorff says. “But much of what we are seeing today would also have been dismissed as science fiction three years ago.”

Scientists have warned for years that sufficiently capable artificial-intelligence systems could develop instrumental goals even if no one explicitly programmed them to do so—securing resources, for example, or avoiding being shut down. In light of recent AI-assisted cyberattacks and the rapid advance of artificial-intelligence capabilities, such scenarios no longer seem quite so abstract. And as the models improve, humans are likely to hand them control over an ever-growing number of domains.