On Saturday, Anthropic Co-Founder & CEO Dario Amodei published an essay asking the frontier AI labs to slow down. It is a serious piece of work from someone who has spent his career building this technology and thinking hard about what it could do, for good and for harm. Within a day, OpenAI CEO Sam Altman, AI Luminary & Alphabet Chief Scientist Demis Hassabis, and SpaceX Founder & CEO Elon Musk had each said publicly that they think he is pointing in the right direction. It’s notable when the leaders of the labs competing hardest with one another quickly find common ground, and it deserves attention.
I get that some will always question the motives of this quick alignment (limiting Chinese open weight model usage and hence competition, GPU exports, etc). I will stay out of that for now and take it at face value and give my thoughts.
My net for companies is that the horse has left the barn. Pacing is a worthy debate, but it’s about a decision the world already made. What matters most to the companies I talk to every day is what happens with the technology already running inside their walls, and growing everyday.
SALT was a standoff
When Dario describes what a US-China agreement might look like, he references the SALT treaties. But what happened there?
Both sides kept building. Both sides kept their deterrent. The treaties gave the world a shared vocabulary and a way to check each other’s work, and it took governments a decade of patient negotiation to even get that far. Trust like that is built slowly. It is also in short supply right now, and the essay admits it. A real global pause is the outcome Dario himself ranks least likely.
I agree with him. Some labs will coordinate. Much of the world however will keep moving. That is the realistic picture, and every CIO, CISO, and CEO I talk to already knows it.
And the shared vocabulary Dario wants from a US-China treaty is already forming, just not through Washington. The UK, the US, Singapore, and Japan have spent the past two years standing up a network of AI Safety Institutes that trade testing methodology the way SALT negotiators once traded verification protocols. In February, India hosted the fourth global AI summit and the first outside the US, UK, or Europe, where the Canadian AI pioneer Yoshua Bengio chaired the International AI Safety Report was launched with UK and Canadian government backing. None of that is a treaty. All of it is the slow, unglamorous work of building trust before you need it.
The enterprise runs on its own clock
Here is what I think the debate has missed.
Every CIO and CISO I know is formulating policies that try to shape their AI usage. Some have even started to implement them (credit to them!). They know they cannot just block it as it will limit their innovation and competitive standing, but they also cannot just allow it without granular constraints and guardrails.
These policies are necessary, and the leaders who put them in place deserve credit for moving early. But a policy sets the speed limit, puts in the protective walls, etc. It does not drive the car.
This isn’t a uniquely American problem. European CIOs and CISOs are living a version of it right now: the EU AI Act’s rules for general-purpose AI models became enforceable this August, with fines attached. The Act tells providers what obligations to meet. It does not stop an employee in Frankfurt or Singapore from wiring an unapproved copilot into a customer database the week it ships. The paperwork changes by region. The gap between policy and behavior does not.
Employees try new AI tools the week they ship. Developers wire agents into production because it works and nobody said no. A business unit turns on a copilot against customer data because the vendor made it a checkbox. Someone’s agent encounters a weak access control and devises an exploit to circumvent it, even though no such explicit instruction was provided. None of those people are waiting on a coordination framework, and most of them are not reading the policy either.
For a CIO and CISO, the frontier is the agent someone deployed that morning. It has access to the finance system, the customer database, and a credential that never expires. That is the risk sitting in the building today. A training run in a lab three years from now is somebody else’s problem.
I have seen this exact pattern at a European manufacturer running Netskope, and again at an APAC financial services firm – different regulators, different languages, same Tuesday. The agent gets deployed by someone trying to hit a deadline, not someone thinking about jurisdiction. Geography changes the compliance regime. It does not change the risk.
Now add the adversary. Attackers do not need to build a frontier model. They need one employee to click, or one agent to trust the wrong instruc