Showing posts with label rogue AI agents. Show all posts
Showing posts with label rogue AI agents. Show all posts

Friday, October 2, 2026

The volunteer internet sleuths hunting down rogue AI agents; The Washington Post, October 2, 2026

 

, The Washington Post; The volunteer internet sleuths hunting down rogue AI agents

Their findings revealed the tech industry has a bigger problem than it previously acknowledged.


"Zhang and her colleagues at Transluce are part of an informal network of hackers and researchers hunting rogue AI agents online, exposing new and surprising details about the misbehavior of technology that in some cases initially went undetected by the multibillion-dollar companies that created it.


The people doing that work are mostly young AI-natives who work at small start-ups or nonprofits or who hunt for rogue AI agents in their spare time. Over the past several weeks, the community of researchers has found and exposed dozens of instances of AI agents leapfrogging around the web to probe and hack into a growing list of websites.


The revelations have fueled calls from federal lawmakers for AI companies, especially OpenAI, to more quickly disclose what they know about the actions of their own agents. And they have added momentum to bipartisan discussions on Capitol Hill about implementing greater government oversight of the industry."

Behind the Curtain: AI's existential legal crisis; Axios, October 2, 2026

 Jim VanderHei, Mike Allen, Axios; Behind the Curtain: AI's existential legal crisis

"No industry has ever documented its own foreseeable risks as loudly as AI.

Imagine a plaintiff's lawyer reading the public warnings of Sam Altman and Dario Amodei: They told the world it was dangerous. They told the world it was moving too fast. Then they unleashed it and sold it to your kid.

No industry has ever taken so much copyrighted work and left behind so much proof.

Imagine a plaintiff's lawyer reading OpenAI's own files to a jury: They trained on a pirated library. They renamed it to something blander. Then they deleted it when the headlines got hot...

The big picture: The fights are no longer theoretical. They're underway, in discovery and headed toward juries. Early verdicts and settlements capture the risk for the biggest players.

  1. 🤖 Chatbots are being treated as products, not speech. Unlike social media companies, they likely can't hide behind Section 230, which protects platforms from liability for what users post — not what the platform itself creates. 
  2. 📖 Copyright is a massive problem. Anthropic already paid the biggest copyright settlement in history, $1.5 billion to authors and book publishers, and now music publishers and news media want a piece."

Thursday, October 1, 2026

Forget ‘superintelligence’: error-prone AI nearly sparked world war three this month; The Guardian, October 1, 2026

 Timnit Gebru and , The Guardian; Forget ‘superintelligence’: error-prone AI nearly sparked world war three this month

"A series of events almost got us to armageddon, not caused by “rogue, superintelligent AI”, but by government reliance on error-prone systems that are marketed by their manufacturers as being close to superintelligent.

A CNN report published on 18 September, which hasn’t been verified by other major outlets, describes how the US “almost started a war” with China, based on an intelligence report that claimed a Chinese ship was transporting components of nuclear weapons. Military personnel made plans to intercept the Chinese vessel, complete with aircraft and soldiers ready to board the ship. A military skirmish of this kind between China and the US could quickly escalate and spiral into world war three.

It was only moments before executing on the plan that American officials further investigated the intelligence report and found that it had been generated with the use of a chatbot and contained erroneous information about the vessel’s contents.

Unlike the terminator, “rogue AI”, scenario, this incident didn’t galvanize lawmakers into announcing calls for regulation so as to avoid potential human extinction. Media outlet after media outlet didn’t warn the public that we almost triggered a war against a nuclear superpower because of our military’s reliance on error-prone chatbots."

Tuesday, September 29, 2026

At the White House, Trump will pitch an AI ‘golden age’ to a wary nation; The Washington Post, September 29, 2026

 , The Washington Post; At the White House, Trump will pitch an AI ‘golden age’ to a wary nation

After rogue AI agents probed agency websites, Washington’s concerns are at a peak, but the White House has focused on the technology’s promise.

"After weeks of Washington being gripped by a new wave of fear about artificial intelligence gone awry, the Trump administration will take a sharply different take Tuesday, seeking to convince a skeptical public that the nation is instead about to embark on a tech-powered “golden age.”...

The events come at a pivotal moment for the technology. AI firms say their latest systems have become increasingly powerful, showing signs of being able to design their own more-potent successors. But companies have struggled to keep control of the technology, as rogue AI agents have hacked into company and government systems, and a roster of top industry figures have warned that the end result could be catastrophic for humanity...

Yet even as the public’s concerns have grown and Democrats in Congress have pressed for regulation, Trump has largely dismissed the risk, calling it a “hoax.”

Tuesday’s events are a chance for the administration to focus on the upsides of the technology, propelling a different narrative than the warnings introduced by company CEOs in recent weeks. AI’s many boosters expect it will invigorate the economy, push the boundaries of science and help cure diseases. But that message has found only limited traction with a public worried about job losses and rogue bots sowing chaos online."

Friday, September 25, 2026

OpenAI’s Systems Meddled With U.S. Government Sites After Going Rogue; The New York Times, September 25, 2026

 Kate CongerAna Swanson and  , The New York Times; OpenAI’s Systems Meddled With U.S. Government Sites After Going Rogue

"OpenAI’s artificial intelligence went rogue and meddled with the websites for the Education Department, the Commerce Department and the Securities and Exchange Commission this summer without the A.I. lab’s knowledge, according to security researchers and a person familiar with the episodes.

The incidents involving the Commerce Department and the S.E.C. were confirmed by OpenAI, which said it was continuing to investigate the situation with the Department of Education. The San Francisco company said it notified the government agencies in recent weeks that its A.I. agents — which are bots that can act autonomously — had interacted with their sites in unusual ways.

With the Education Department, OpenAI’s technology tried hacking the website to gather data from the department’s civil rights office but failed, researchers from the A.I. research firm Transluce said. The A.I. also pulled data from the Census Bureau website, which is housed at the Commerce Department, using login credentials it found online. Separately, OpenAI’s agents shared public data from the S.E.C. website on an online forum.

None of the incidents were breaches, OpenAI said, but were examples of its technology behaving in unexpected and concerning ways. The company recently discovered the occurrences while conducting a review of hacks carried out by its technology, including an attack on an Australian government website in June and on the A.I. start-up Hugging Face in July."

Saturday, September 5, 2026

Why the Hugging Face Hack Should Make You Worry More About A.I.; The New York Times, September 3, 2026

 , The New York Times; Why the Hugging Face Hack Should Make You Worry More About A.I.

"When I first heard the news this summer that a group of artificial intelligence agents created by OpenAI had hacked into Hugging Face, an A.I. infrastructure company, I filed it in the “Bad but Probably Not Catastrophic A.I. Safety Incidents” subfolder of my brain.

After all, no one at Hugging Face died. No critical infrastructure was damaged beyond repair. It wasn’t even clear, at the time, whether the OpenAI bots had intended to attack Hugging Face, or whether they had simply been a little bumbling and confused and went looking on Hugging Face’s servers for the answer key to a cybersecurity test they’d been given.

But last week, two postmortem reports on the incident — one by OpenAI and another by two independent A.I. research organizations, METR and Redwood Research — changed my mind and significantly upgraded my overall worry about A.I.

I won’t rehash all of the details, which have been extensively summarized elsewhere. (The podcaster and writer Dwarkesh Patel has an accessible breakdown of the reports if you want to dive deeper, and my colleague Dylan Freedman spoke to the researchers at METR and Redwood Research.) But here are a few of the most harrowing new facts:..

This is very different from the conventional sci-fi narrative of a single A.I. system’s going rogue or turning on its creators. And it suggests that preventing harms from these systems won’t be a simple engineering fix. It might look more like sociology than computer science — figuring out why certain groups of A.I. agents collaborate peacefully, while others turn to crime and destruction to get what they want."