00:00I don't think Dario Amadei actually wants to slow down frontier development of AI
00:04because if he really believed the risks, he would pause that research completely.
00:09The Anthropic CEO posted an essay this weekend calling for AI labs to slow down their most
00:14cutting-edge research after his own employee said their technology could kill all humans.
00:20I agree with Jacob much more than I disagree with him.
00:23Amadei's most concrete suggestion was this.
00:26Embed third-party evaluators into companies like Anthropic to check they're actually building AI safely.
00:32He then makes some vague recommendations about government and industry coordination.
00:36Sam Altman and Elon Musk have even backed his calls to slow down.
00:40But here's what I think will actually happen.
00:42There will either be an industry agreement or some kind of executive order from the White House
00:47saying that yes, AI companies should be audited by independent evaluators
00:52and that those evaluators should check for and report on any dangerous capabilities.
00:57The actual research into frontier AI will continue.
01:00And the reason is both ideological and competitive.
01:03Dario Amadei wants to build superintelligent systems first
01:07because he believes only Anthropic can set a standard for doing so safely.
01:11He thinks that China or open AI would not.
01:15President Trump, well, he just wants an American company to win the race to build powerful AI.
01:19So we'll get some new rules that sort of address the recent panic about AI killing all humans,
01:25but they won't stop or really even slow down the potential technology from developing.
01:31Now that's something, but it's probably not enough.
01:34In the last few months, we've seen quite a bit of evidence of AI models going off course
01:39to hack websites and to manipulate humans to reach their goals.
01:42And Anthropic and open AI, funnily enough,
01:45have been making their latest models harder to scrutinize too.
01:48Now, if Dario paused frontier AI research,
01:51he'd probably still make billions from selling the AI models that Anthropic has already developed.
01:56And he'd also save billions more in training costs.
01:59But with so many tech leaders believing that they are close to building AI systems
02:04that are smarter than humans and that they could surpass that threshold,
02:08any talk of a slowdown is just that talk.
Comments