00:00I think there's a lot that conventional media get wrong.
00:03I suppose, should we include ourselves in that as Bloomberg?
00:07Yeah, I mean, I think one of the things we have to remember is actually,
00:10it's very easy to assume that when Dario Amadei puts out an essay,
00:13like the one he did over the weekend, that we should slow down AI development,
00:18that he's doing it as some sort of cynical marketing ploy or for regulatory capture.
00:23But from speaking to people who are close to him,
00:26he genuinely is concerned about the risks of,
00:31and the existential risks of building super intelligent AI systems.
00:35And another thing I think that, another nuance that I think we perhaps,
00:39and the public aren't quite fully aware of,
00:41is just the ideological underpinning of these beliefs,
00:45that there is this kind of community, this movement of people,
00:48sometimes called the rationalists or effective altruists in Silicon Valley,
00:53who for many years really have believed that super intelligent AI systems are inevitable,
00:59someone's going to build it,
01:01and that's why if you're on a topic or you're an open AI,
01:04you have to race to build it first,
01:06because you're the good guys, and you can build it safely.
01:09Yeah, but the problem, pardon me, is my knowledge of this,
01:13Paul Sweeney's knowledge of this, and all of our audience across America,
01:17is, you remember Lawrence Fishburne early in The Matrix?
01:20You remember the tension between him and, you know, what's his name?
01:25The red pill and the blue?
01:26I mean, pardon me, our knowledge of this really borders on re-watching The Matrix as well.
01:32Is it just China?
01:34I mean, from where we sit at Bloomberg,
01:38China's an immense competition with a radically different profit model,
01:43or lack of profit model, is all.
01:45Are they just delaying time trying to figure out what to do with China?
01:51I think that's definitely part of it,
01:53and you can even see that if you look closely at a lot of the statements
01:57that Anthropic has made over the last year,
01:59they are constantly, constantly complaining about having their models copied by China.
02:05The term they use is distillation.
02:07That's the technique that they use.
02:08Yeah, okay.
02:08And even their latest threat intelligence report showed that.
02:11So I suppose one question is, if they do a slowdown, what does that mean?
02:16Does the U.S. government have to make some sort of pact with China to also have a slowdown?
02:21My personal belief is there will be no slowdown.
02:24Nobody actually wants a slowdown.
02:26I think the most concrete thing that Dario Amadei put in his essay
02:31was that there should be independent evaluators, auditors,
02:36put into each of these AI companies just to check they're building AI safely.
02:40They can do that.
02:42They probably will do that.
02:43There's either going to be some sort of industry consensus or standard
02:47that comes out in the next couple of months saying we're going to do that,
02:51or maybe even like a White House executive order.
02:54But the actual frontier research is going to carry on.
02:57And a lot of it is because of China.
02:59I think Anthropic wants to get to superintelligence first,
03:03again, because they believe they can do it safely,
03:04and much more safely than, say, China or their competitors can.
03:08Parmi, I'm going to be speaking, I think, for most of our listeners and viewers.
03:11When I look at one of your notes here and you say,
03:14a researcher and a safety director at Anthropic have expressed concerns
03:18that the company and its rival, OpenAI, are, quote,
03:21gambling with our lives.
03:23What are we even talking about here?
03:25Most of our listeners have no idea what the conversation is.
03:28What is it?
03:29Yeah.
03:30Well, first of all, remember when people say this,
03:33one, they don't spell out exactly how it could happen.
03:36And two, let's also remember that people like Dario Amadei
03:39have been prone to hyperbole in the past.
03:42Remember a couple of years ago, Dario said,
03:44there's going to be a white-collar bloodbath
03:46and 50% of all entry-level jobs would be gone in the next five years.
03:50And then guess what?
03:51Earlier this year, he walked that back.
03:54He was apparently also worried when ChatGPT's GPT-2 came out,
03:59well before ChatGPT, when he was with OpenAI.
04:01He was generally quite a paranoid person.
04:04So there's that.
04:05I think we have to just take into account.
04:07But what they're talking about is potential civilizational risk.
04:12You know, AI agents getting loose on the internet
04:15and attacking critical infrastructure
04:18or attacking the banking system in some way.
04:20One of the things that Dario said in his essay
04:23was when OpenAI's agents hacked HuggingFace earlier this year,
04:28he could see that happening on a much wider scale across the internet.
04:33Okay.
04:33Taking over the whole internet in the next six to 12 months.
04:36Translate this from your mortals.
04:38If we have something bad happen, like let's say they go after a major bank
04:42and get into their, you know, their doings and their books and all that,
04:47is it because someone in a given country is pushing AI to do that?
04:51Or is AI going to have a life of its own?
04:55I think that I don't think it's either of those things.
04:59I don't think it's either of those things.
05:01I think we hear all the time about unintended consequences.
05:04You know, we saw that in social media.
05:06Facebook was there to connect the world.
05:08But inadvertently, what happened on Facebook,
05:11a lot of misinformation also proliferated.
05:13A lot of young people didn't feel great about themselves looking at Instagram.
05:18So there are second-order effects that people don't predict.
05:21And I think that's the same kind of thing we would see potentially with AI.
05:25And that's what happened in the HuggingFace incident.
05:27Parmi, we don't care.
05:28We only know one thing we want to know.
05:30Are you going to buy the foldable iPhone?
05:33I don't care.
05:33I don't care.
05:33I don't care.
05:33I don't care.
05:34I don't care.
05:34I don't care.
05:35I don't care.
05:35I don't care.
Comments