AI & Machine Learning
Business Insiderabout 3 hours ago
0

The people warning us about AI are also building it

AI

Anthropic, an AI safety advocate, faced a US export ban on its advanced models, highlighting the tension between safety concerns and competitive pressures in the AI industry.

The people warning us about AI are also building it

Intelligence Insights

Context + impact, normalized for TechCulture.

The Big Picture
Anthropic, an AI company known for its safety advocacy, had its advanced models Fable 5 and Mythos banned by the US government over security risks, forcing it to disable them globally. This incident underscores the conflict between AI safety and business competition, as Anthropic has both warned about AI dangers and pushed to commercialize powerful systems. The company's history shows a pattern of safety commitments being weakened under market pressure, such as in February when it reduced foundational safety pledges. The export ban reflects governments' struggle to regulate technology that even its creators don't fully understand, while fearing that slowing innovation could cede advantage to rivals. The core issue remains unresolved: who decides what level of AI risk is acceptable, with some experts suggesting more AI oversight itself as a solution.
Why It Matters
This article highlights the fundamental conflict at the heart of the AI industry: the same people warning about AI's dangers are the ones racing to build and profit from it. Anthropic's struggle to balance safety commitments with competitive pressures shows how market forces can undermine even well-intentioned safeguards. The standoff with the White House over export controls reveals that governments are equally conflicted, fearing both the risks of advanced AI and the economic consequences of slowing down. Ultimately, the debate isn't about whether to regulate AI, but who gets to decide what 'safe enough' means—and no one wants to make the first move.

Deepen your understanding

Use our AI to break down complex signals.

Select an AI action to generate more depth.

Anthropic's Dario Amodei
Anthropic
Anthropic's Dario Amodei

Bloomberg/Getty Images

Anthropic has pushed for better safety protocols on AI. Now it's getting a firsthand view of the impact that can have on a business.

The AI giant's new models were sidelined by the White House over potential security risks. BI's Natalie Musumeci has a cheat sheet on the saga, which kicked off on Friday.

My colleagues at Politico also have a breakdown on the whirlwind 24 hours leading up to the White House's clampdown on Anthropic.

For Anthropic, it's been a hectic year balancing concerns over AI safety with its desire to compete in the AI race.

Just look at the back-and-forth over the past six months:

Jan. 27: Anthropic CEO Dario Amodei drops a 19,000-word message on the future of AI and the "serious civilizational challenge" it poses.

Feb. 24: Anthropic weakens its foundational safety commitment amid heightened competition and a lack of government regulation.

Feb. 27: A dispute between Anthropic and the Department of Defense over how its AI models are used leads the DoD to label Anthropic a supply chain risk.

April 7: Anthropic said its Mythos model is too powerful for the public, citing its knack for finding "high-severity vulnerabilities."

June 1: Anthropic files a confidential S-1 for its IPO.

June 5: Anthropic calls for a coordinated slowdown among frontier AI labs "to enable societal structures and alignment research to keep up with the advance of the technology."

June 9: Anthropic releases Fable 5, which it built by putting safeguards on the aforementioned Mythos model.

June 10: Amodei publishes a blog saying AI is moving at a "lightning pace" while policy is "moving very slowly."

June 12: A US order barring foreign entities or individuals from using Anthropic's Fable 5 and Mythos leads it to disable the models for everyone.

June 15: Trump officials reportedly met with Anthropic to resolve the export ban on Fable.

The Mythos drama is the clearest example yet of the difficult position the AI industry finds itself in.

The people most qualified to warn about the dangers of advanced AI are also the ones who stand to make trillions creating it.

Governments, meanwhile, are left trying to regulate technology that the people building it don't fully understand. And just like tech executives, they're worried about competition. Slowing innovation means risking your entire country falling behind.

That's the tricky part. You all might agree that something needs to be done, but no one wants to be the first to actually do something about it.

The real challenge isn't building safer AI. It's figuring out who gets to decide what "safe enough" means.

Many experts agree on the best way to maintain oversight of increasingly complex AI, but you probably won't like it.

It's more AI.

Read the original article on Business Insider
Big Tech AI Cybersecurity Policy

Intelligence Exchange

0

Log in to participate in the exchange.

Sign In

Syncing Discussions...

Finding Related Intelligence...