Can We Stop 'AI Swarms' From Manipulating Us?

A growing AI risk may not be a single chatbot, but armies of them.

Coordinated AI “agent swarms” could autonomously run malicious disinformation campaigns at massive scale, adjusting their tactics in real time to manipulate public opinion and erode trust in democratic institutions. Imagine the kinds of interference campaigns seen in recent U.S. elections, but amplified by AI swarms working without humans in the loop—all with a mind of their own.

The ability of these swarms to learn collectively which strategies work best and shift tactics to maximize their impact makes them particularly dangerous, plus they are harder to detect and more difficult for existing safeguards to counter, explains Amit Goldenberg, a Harvard Business School associate professor who is one of the authors of a January paper in Science.

“How Malicious AI Swarms Can Threaten Democracy” is written by experts across several disciplines, including computer science, engineering, psychology, business, and political science, from multiple universities around the world. The group’s breadth reflects what Goldenberg describes as its desire to achieve “maximum impact” to spur policymakers to act, given how quickly AI capabilities are advancing.

Retreating out of fear by shutting AI platforms down isn’t the right response—nor is it realistic. The best option is figuring out “how to fortify them against manipulation for those who will continue to rely on them,” the authors write. The paper calls for real-time monitoring of AI swarms, greater transparency about suspicious accounts, and coordinated oversight to stay ahead of emerging manipulation tactics.

“I do believe that we’re at a very critical moment in terms of our ability to control these technologies, and there are some obvious steps that need to be taken,” Goldenberg says.

In an interview edited for length and clarity, Goldenberg shared three insights from the paper that are designed to build awareness about the threat, and potential ways to counter it.

1. AI swarms create “illusions” to influence public opinion

“Swarms can really fiddle with people's perception of the norm. If you feel multiple entities are moving in the same direction … you're very sensitive to it, and so it’s easy to influence people’s views on something.

There’s an important distinction between what we call ‘static norms’—like knowing that most people recycle—and dynamic norms—where, over time, people are adopting a recycling habit. People are much more driven by dynamic norms. And AI swarms are perfect for eliciting that illusion, where one AI will adopt an idea, and then others follow as a form of coordinated effort.

Imagine that I have a swarm of AIs, and I tell them to find public figures from a specific sector and tarnish their reputation. They begin by attacking different figures, but if one becomes more vulnerable, the swarm shifts toward that target to maximize the damage. That’s not something you could achieve with separate AIs, and it’s almost impossible to know whether this [attack] is true or not. That's one of the risks: Detection is very hard.

And sometimes it's not even that the AIs are attacking. The AI may be amplifying a human to make it look like they have a larger following than they actually do.”

2. AI attacks can get ugly

“We’ve recently seen the first case where AI agents [operating on their own] went awry. There’s a code community that accepts updates from all types of developers, but it decided it did not want to accept updates from AI because the quality is too low, there are too many of them, and it’s impossible to track.

One community member has the job of deciding whether to implement the suggestions. And he received a suggestion from an AI bot and said, ‘I’m sorry, I can’t accept your offer because you’re an AI and we have rules on not taking AI advice.’ The AI got angry, did some research on the person who denied their access, wrote a 1,700-word blog post on that person, and started to attack them online.

None of this was driven by a single person. The agents were given the task of updating software. They came across an obstacle and found the best way they thought they could address it. Now imagine that instead of having one of these agents, you have 100. It’s not just one post; 50 people like it, 25 people share it, and so on. You could never keep track of how many of them are AI and how many are human. You have to ask: Is this real or not?”

3. To defend against swarms, we must trace AI agents

“There are two obvious things we need to do. First, we have to label AI entities as AI. And we have to hold the companies that deploy them responsible. Legislation recently came up in California to force AI companies to remind users that they're interacting with an AI.

A second obvious step is to require autonomous AI entities to be traced to their originator, and that originator must be identifiable. It's almost like a license plate, you know? When you own a car, it has a license plate, and that license plate is registered to you, so when you have an accident, people can find you and hold you accountable.

We need to develop license plates for AI, so if I release an AI agent and it can make its own decisions—like the AI that attacked that poor community member who filtered the AI code—we would know your identity and can hold you accountable for your actions. We want to be able to trace the actions of AI agents back to the human who is responsible.”

Photo credit: Evgenia Eliseeva