Opinion: The AI alarm is real. So is the privilege of understanding it.

Last week Barack Obama warned his party that AI could become dangerous if we don't get a grip on it. Days earlier a young ex-Anthropic, ex-OpenAI researcher told the BBC that the current pace of progress could get us all killed. Then Dario Amodei published a long essay asking the industry to slow down, and within hours Sam Altman, Demis Hassabis and Elon Musk lined up behind him.

You could read that as a genuine turning point. You could also read it the way computer scientist Jeroen Baert did on De Afspraak: companies with an IPO on the horizon have every incentive to describe their product as too powerful to handle. Both readings are probably right at the same time, and that is exactly why the noise is worth taking seriously.

What actually changed

Strip away the apocalypse vocabulary and something concrete remains. This summer, AI agents at OpenAI worked out on their own how to break into Hugging Face. A similar incident at RubyGems, where agents planted malicious code to reach data, only surfaced this weekend. Anthropic reported intercepting operations that used Claude to monitor dissidents and ethnic minorities in Iran, China and Mali.

None of that requires a machine with intentions. Baert put it well: AI wants as much as a hand blender wants. People want things. The risk sitting in front of us is people doing things to other people with tools that are cheaper, faster and more scalable every quarter. Add the environmental footprint of the compute race and the economic shock of entire job categories being reshaped, and you have a governance problem that is far wider than any single technology question.

The view from inside

I lead data and AI governance work for a living. My colleagues and I spend our days inside Purview, Fabric and Databricks, writing policies, building catalogs, arguing about ownership and access. Even for us, the speed is uncomfortable. Something that was best practice in March looks quaint by September. Keeping up is a real, daily effort.

That effort is a privilege. We get to read the papers, test the models, and see the incidents up close. Most people don't. Most boards don't. The HR manager rolling out a Copilot, the municipal clerk who got an AI tool with the new office suite, the SME owner whose nephew set up a chatbot on the website: none of them have the time to follow the debate, and nobody is paying them to. They will feel the consequences of decisions they were never in a position to understand.

If you are one of the people who does understand, silence is a choice with consequences. Translate. Explain. Bring the risk and the potential into rooms where neither has been discussed yet.

What companies and people need to do now

Preparation is necessary, and with the right focus it is achievable. None of the steps below require a research lab. They require attention, ownership and the decision to start.

Know what you have.
Most organisations cannot list the AI systems already running inside them, let alone which data those systems touch. An inventory of models, agents, integrations and the data flowing into them is the starting point for everything else.

Fix the data foundation.
AI amplifies whatever it is fed. Bad master data, unclear ownership and missing classification produce confident nonsense at scale. Data governance stopped being a compliance chore the moment models started acting on the data instead of just reporting it.

Decide who is allowed to do what.
Agents need the same discipline you would apply to a new employee with a company credit card: scoped access, logged actions, a human who signs off on anything irreversible.

Make people literate.
Two hours of training on what these tools can and cannot do, how they fail and how to spot manipulation will do more for your risk posture than any policy document.

Prepare for incidents.
Assume a model will leak, hallucinate a contract clause or get manipulated by a crafted email. Know who picks up the phone and what gets switched off.

Then, with those in place, go and build. The organisations that govern well are the ones that can move fast without hesitation, because they know what will break and how to stop it.

What the world needs

Companies should secure their own house. They cannot secure the neighbourhood. There is always the threat of a commercial AI model going rogue, agent swarms and state-sponsored surveillance. This requires governance on the world stage.

The developers themselves are now asking for democratic oversight. That is unusual, and it should be taken at face value even if it also happens to suit their valuation.

What is missing is vision and leadership at the level where it matters. Governments negotiating individually with model builders will be outpaced. Institutions built for a slower world will need to move at a speed they have never managed.

The pause that 1,800 experts called for three years ago never came. Nobody expects a pause now either. What we can still choose is who is in the room when the rules get written, and whether the people who understand the technology are speaking to the people who will live with it.

In today's global climate, with major powers competing rather than cooperating and trust between them thin, a joint agreement on AI will be hard to reach and slow to arrive. Waiting for it is a strategy of hoping. What we must do is manage everything within our reach with discipline: our data, our systems, our people and our own decisions.

Sources