If We Know AI Could Hurt Us, Why Can't We Just Slow Down?

If you've been following AI closely, the strange thing about the current wave of alarm isn't necessarily what is being said.

It's how suddenly everyone seems to be listening.

In May 2023, hundreds of researchers and technology leaders signed a statement from the Center for AI Safety arguing that mitigating the risk of extinction from AI should be treated as a global priority alongside pandemics and nuclear war.

The signatories included Sam Altman, Demis Hassabis and Dario Amodei, leaders of three of the organisations at the forefront of developing increasingly capable AI systems.

More than three years later, the warnings have become increasingly difficult to miss.

Amodei recently argued that we need to "pace the frontier", citing risks including loss of control, cyberattacks, biological misuse and economic disruption, while also making the case that sufficiently capable AI could deliver extraordinary benefits in medicine, science and economic growth.

That combination is important.

The argument isn't necessarily that AI is bad.

It is that something can be extraordinarily useful and extraordinarily risky at the same time.

And that creates a rather awkward question.

If the people building increasingly powerful AI systems have been warning us about the risks for years, why don't we just slow down?

We are still pressing the accelerator

There's a tendency to talk about AI progress as though it were a natural phenomenon.

The technology is advancing. Models are getting more powerful. AI is coming.

But AI development isn't weather.

Humans are designing the systems, building data centres, buying chips, training models, deploying products and investing enormous amounts of money in making them more capable.

That doesn't mean progress is predictable or that any individual person can simply turn it off.

But it does mean we shouldn't surrender human agency too quickly.

It's also worth being careful with the word "better".

A system can become more capable, efficient or effective without becoming "better" in any meaningful human sense.

An AI recruitment system could become more efficient at identifying patterns in historical hiring data while simultaneously becoming more efficient at reproducing a historical bias.

Capability and desirability aren't the same thing.

Nor does AI simply become exponentially more capable because we keep feeding it data. Progress depends on compute, algorithms, data quality, training techniques and many other factors.

But there is a potentially important feedback loop emerging.

As AI becomes more capable, we're increasingly able to use AI to help develop AI: writing code, analysing experiments, generating synthetic data, running evaluations and assisting researchers.

In other words, we're pressing the accelerator harder partly because the car is helping us design its next engine.

That doesn't require a science-fiction scenario in which an AI has escaped from a laboratory and secretly started improving itself.

Human competition, combined with increasingly capable AI-assisted development, could be enough to make the pace difficult to govern.

So why doesn't somebody ease off?

This is where the problem becomes less technological and more human.

Imagine two AI companies.

Both believe that developing increasingly powerful systems carries genuine risks. Both would prefer stronger testing and safeguards.

But Company A decides unilaterally to slow development for twelve months.

Company B doesn't.

Company A has now potentially handed its competitor a year-long commercial advantage.

Scale that problem up from companies to countries and it becomes even harder.

A government may believe that frontier AI requires stronger controls while simultaneously worrying that slowing its domestic industry could allow another country to gain an economic or strategic advantage.

This is a classic collective-action problem.

Suddenly "why don't we slow down?" becomes a much more difficult question.

Everyone can see the brake. Nobody wants to be the first to press it.

What happened at the White House this week is fascinating

On 29 September, President Donald Trump met leaders from some of the world's largest technology companies at the White House.

Among them were Anthropic CEO Dario Amodei, Google CEO Sundar Pichai, Meta CEO Mark Zuckerberg, Nvidia CEO Jensen Huang, Elon Musk and OpenAI president Greg Brockman.

They signed a voluntary agreement establishing safety commitments for frontier AI systems.

The accord calls for internal controls and monitoring, dedicated internal assurance, independent external evaluation and board-level oversight. The participating companies also agreed to meet regularly to develop safety standards and best practices.

It is voluntary rather than legally binding, although the agreement leaves open the possibility that some measures could eventually become regulation.

There is plenty to debate about whether voluntary commitments are sufficient.

But I find the underlying idea more interesting.

They haven't agreed to stop the race. They've agreed to put barriers around the track.

That may turn out to be a much more realistic model for AI governance than imagining the world's leading companies and governments collectively agreeing simply to stop developing more capable systems.

It is deliberate risk management rather than prohibition.

The warnings aren't new. The attention is.

None of this means today's concerns should be dismissed as hysteria.

Capabilities have changed. The technology has become more widely deployed. AI systems are being given access to more tools and used in more consequential settings.

But it's worth remembering that warnings from within the AI industry did not suddenly appear in 2026.

That matters when interpreting the current media conversation.

My instinct is that dramatic AI stories naturally attract attention.

"AI could kill us all" is, unsurprisingly, a rather stronger headline than "researchers continue discussing a complicated spectrum of low-probability, high-impact risks".

But I don't think we need to assume cynical motives to notice that distinction.

The more interesting question is why the warnings are resonating so strongly now.

Perhaps it is because AI has moved from something people read about to something millions use every day.

Perhaps it is because increasingly autonomous systems make abstract risks easier to imagine.

Perhaps it is because the warnings themselves have become more urgent.

Most likely, it is some combination of all three.

Regulation creates another awkward problem

There is another side to this debate that shouldn't be ignored.

If complying with sophisticated AI safety regulation requires expensive evaluations, specialist teams, auditors and extensive technical infrastructure, the largest AI companies are also those best equipped to absorb those costs.

Smaller competitors may not be.

That doesn't make safety requirements wrong.

But good governance has to consider unintended consequences too.

Frontier AI development already has significant barriers to entry, including access to compute, capital and specialised talent. Poorly designed regulation could add another.

So we end up with another difficult balancing act.

Too little governance may create unacceptable risks.

Badly designed governance could entrench the companies that already dominate the technology.

The answer cannot simply be "regulate AI" any more than it can be "let innovation run".

The details matter.

The same problem exists inside ordinary businesses

All of this can feel rather remote if you're running a 50-person company rather than an AI laboratory.

But shrink the scale and the underlying problem looks surprisingly familiar.

We know we should test this properly, but our competitor has already launched it.

We know employees are using AI tools we haven't approved, but they're getting more work done.

We know this system occasionally produces unreliable results, but adding human review will make the process slower.

That's the same tension.

Commercial pressure starts determining risk appetite by default.

And that is where governance becomes useful.

Not as a bureaucratic mechanism for preventing people from using AI, but as a way of deciding deliberately where an organisation is prepared to move quickly, where human oversight remains necessary, what should be monitored and where the consequences of failure are simply too high to take the chance.

The White House accord, whatever ultimately comes of it, follows a recognisable governance pattern: controls, monitoring, independent evaluation, oversight and remediation.

Those principles scale surprisingly well.

An SME obviously doesn't need the governance machinery of a frontier AI laboratory.

But it does need to know what AI it is using, what could go wrong, who is accountable and what happens when something does.

Maybe slowing down is the wrong question

I've become less convinced that the most useful debate is whether humanity should "stop AI".

It creates a binary choice that probably doesn't exist.

The more practical question is how we retain the benefits of moving quickly without allowing competitive pressure to decide which risks we're prepared to accept.

There will be people who think current safeguards go far too far.

There will be others who think voluntary commitments are nowhere near enough.

And there remains enormous uncertainty about how capable future AI systems will become and what risks they will actually create.

But one thing seems increasingly difficult to argue.

We cannot simultaneously acknowledge potentially serious risks and then treat the speed of development as something happening entirely beyond human control.

We're still building it.

We're still funding it.

We're still deciding where to deploy it.

And we're increasingly using it to help us build what comes next.

The problem may not be that nobody can see the brakes.

It may be that nobody wants to be the first to press them.

Sources and further reading

• Center for AI Safety, Statement on AI Risk, May 2023
• Dario Amodei, We Must Pace the Frontier
• Al Jazeera, Trump, tech bosses sign voluntary pact pledging 'robust' AI safeguards, 29 September 2026
• The Guardian, Trump announces 'morally binding' AI deal among tech CEOs, 29 September 2026
• UK Government, Frontier AI: capabilities and risks (discussion paper)