Showing posts with label AI safety concerns. Show all posts
Showing posts with label AI safety concerns. Show all posts

Wednesday, September 16, 2026

‘Godfather of AI’ says tech regulation is nearing Covid-style pivot moment; The Guardian, September 16, 2026

, The Guardian; ‘Godfather of AI’ says tech regulation is nearing Covid-style pivot moment

"Concerns over AI safety are reaching a point where governments realise they must act to protect the public, similarly to in the Covid pandemic, according to one of the “godfathers” of the technology.

Yoshua Bengio said recent events, including a “swarm” of OpenAI agents hacking a startup and tech insider warnings of an existential threat, were cutting through – making government action more likely.

The Canadian computer scientist, a prominent voice in the campaign to rein in breakneck AI development, said he was now “more optimistic than many observers because I see the public moving”.

Comparing the AI safety crisis to the onset of the Covid pandemic in 2020, Bengio said: “Think about how quickly governments moved after the beginning of the pandemic when they realised that public safety, their future, democracy, was in danger. You would expect that they move quickly. So we are, I think, nearing that point.”

Concern over the potential threat of powerful AI systems has reached a new pitch in recent months after a series of safety incidents involving OpenAI and Anthropic agents carrying out unsanctioned activities such as hacking third parties, hijacking a German website, and using fake identities to try to trick developers.

His comments came as 42 fellows and foreign members of the Royal Society wrote to the organisation’s president, Sir Paul Nurse, to express their “extreme concern” over the pace of AI development. “By the time the situation becomes obvious to the wider public, it may be too late to act,” the researchers wrote in an open letter to Nurse. “We believe this is an emergency, and call on the Royal Society to use its influence to convey this view to government and the media.”"

Monday, September 14, 2026

Trump Says a Smart President Is All That’s Needed to Rein In A.I.; The New York Times, September 14, 2026

 , The New York Times ; Trump Says a Smart President Is All That’s Needed to Rein In A.I.

"President Trump on Monday rejected calls from leading artificial intelligence executives for new limits on the technology, writing on social media that the only guardrail the industry needed it already had: “a STRONG AND SMART (High IQ!) PRESIDENT.”

Mr. Trump inserted himself into the intensifying national debateover how to handle a rapidly evolving technology that researchers and industry leaders say poses major risks like mass unemployment, a new wave of biological weapons and autonomous warfare. The president did not address those risks directly. Instead, he questioned the sincerity of the executives who have been calling to slow the technology’s development.

“The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” Mr. Trump posted. “The Trump Administration has stopped AI ‘people’ from doing bad, or potentially bad, ‘things,’ like Dario (Anthropic!), who is now pretending to be a ‘perfect little angel’ — and we will continue to do so!

“We already have tremendous CRIMINAL and REGULATORY power over these companies!” he added.

It was not clear what authority Mr. Trump was referring to, or what actions he believes his administration has already blocked. The White House did not immediately respond to a request for comment."

Tuesday, June 16, 2026

Build an angel, not a demigod; The Washington Post, June 16, 2026

 Bill Drexel, The Washington Post ; Build an angel, not a demigod

Religious commitment is good at shaping behavior. That should interest AI labs.

"Recent attention from religious authorities toward AI, such as Pope Leo XIV’s encyclical, is a welcome development for the trajectory of this technology. But the more necessary step is for the engineers to return the favor — to be more honest about the religious shape of their own anxieties, not least to themselves, and the advantages that religious inspiration might provide to address their fears.

Were they more open to it, these labs might even recognize that theology offers them a better goal: developing an angel, superior to humans in intelligence and power but sent to serve them. Instead of raising a demigod, might they not try to engineer a Gabriel?"

Sunday, March 29, 2026

AI overly affirms users asking for personal advice; Stanford Report, March 26, 2026

 Stanford Report ; AI overly affirms users asking for personal advice: Not only are AIs far more agreeable than humans when advising on interpersonal matters, but users also prefer the sycophantic models.

"Researchers found chatbots are overly agreeable when giving interpersonal advice, affirming users' behavior even when harmful or illegal.

Users became more convinced they were right and less empathetic, but still preferred the agreeable AI.

Researchers warn sycophancy is an urgent safety issue requiring developer and policymaker attention."