00:00The six incidents that are laid out by OpenAI are kind of further confirmation that basically
00:06AI companies are seeing these really unintended and sometimes alarming behaviors that advanced
00:13AI can do during the training and evaluation process. So while these incidents didn't actually
00:18breach or hack an external app like the hugging face incident, OpenAI says, they did show that
00:25these agents are sharing files that are not supposed to, are giving themselves or itself
00:31instructions on how to disregard human leadership. And so that you can find that even if that's not
00:40actually causing a hack in the real world, that kind of behavior gives a moment for pause and
00:46concern even by the AI lab's own admission of and raises the question of do they really have control
00:52over the agents that they're creating? Do they have control? And I read from the statement here,
00:56we do not believe that the AI industry has solved alignment and monitoring to a sufficient degree
01:01to continue responsibly scaling at maximum speed for much longer. Could you just talk about the
01:07difficulty of keeping track of all of this, for lack of a better phrase, just being able to note
01:11when these incidents happen and to stay on top of when things like this might occur? Yeah. So if you
01:17think about the field right now and the direction it's going in, it's having many, many, many,
01:22many, many thousands of agents going after any one, you know, set of research problems at a time.
01:31Anthropic, OpenAI's, you know, chief competitor in a report yesterday that they published said that
01:36they have 30,000 AI agents in a period of time that they were monitoring just on R&D. So
01:44if you think
01:45about that, now these labs have to go monitor, well, what are these tens of thousands of agents that are
01:51doing research for us actually doing out there in the world at any one time? And just monitoring that,
01:58just observing that kind of behavior is a huge undertaking. And so when OpenAI makes statements
02:04like we haven't solved the alignment problem yet, what they're saying is that we can't be sure,
02:09essentially, that these AIs that we are testing are really taking action in line with what humans
02:15want the AIs to do. The memo from Dario Amadei continues to resonate here a week out. And one
02:22of the things he suggested was there be independent monitors in these companies. As you talk to experts,
02:27how critical is that to kind of widen the aperture to get more people to have visibility to what's
02:32happening inside these companies? Yeah, I think it is a very, you know, there is a very strong push
02:41right now from, you know, these are employees who are concerned within the labs. These are lab leaders
02:49and, you know, some external safety advocates. You see people like the godfather of AI, Jeff Hinton,
02:56for example, I saw just calling for this group of outside auditors essentially to come in and test
03:05these AI models. Now, the big question is going to be, are these auditors truly independent?
03:11Where are they getting their funding from? And also, what kind of viewpoints are represented there?
03:16Will these auditors, you know, will they have a lot of power to either, you know, assess positively or
03:22negatively the AI potentially speed up or slow down the AI race? Well, people want to make sure that
03:27the types of decisions that these auditors are making are fair and that the people behind them
03:32are as unbiased as possible. So we're on the precipice of this major summit, President Xi is
03:38coming to Washington to meet with President Trump. And I think going into that, as of just a few days
03:42ago, the thinking was, oh, this would be about trade. This would be about economic policy. But it now
03:46really seems like AI is going to be front and center there. How do you look at the potential import
03:51of
03:51that event, these two leaders getting together amidst what we've seen suggested here in recent
03:55days, that this is a moment when, you know, perhaps China and the United States can come
03:59together to kind of chart a course in concert with one another about how they should approach AI?
04:06Yeah, I think that if you think back to even two years ago, even a year ago, AI was not
04:12as front and
04:13center of the political debate. But right now, we have this kind of triple whammy of one, there are these
04:18imminent safety concerns that are becoming more real because of cyber hacking and the AI's advanced
04:24cyber hacking capabilities. There is the anxiety about job loss and everyday people wondering what's
04:31going to happen if AI takes my job. And also, conversely, what effect does that have on businesses?
04:37And then there is the third kind of bucket of fears around just, well, are these AIs actually
04:45helping us sort of in terms of the way that we're communicating with each other? We see things
04:51like people, you know, suing the AI company saying that they're reinforcing delusional beliefs and
04:58things like that. So while that may be a smaller subset of people who use AI, much like with social
05:02media, I think there's also this concern about, well, how is this actually affecting our mental,
05:08emotional well-being, especially for those who are vulnerable or children, et cetera. So I think
05:12because of all of that, oh, and then a fourth big bucket I should mention, maybe top of mind for
05:16most people is data centers, right? You also have concern around data center build out. So I think
05:21for all of those reasons, AI has become something that people like President Trump are not going to
05:25ignore, right, or are not going to just say we're, you know, it's hard for any politician to be
05:32completely just letting AI, just unaddressing these topics. That being said, Trump has positioned
05:37himself as very pro AI. He doesn't want to lose to China. And so what will be key here is
05:42if the U.S.
05:42and China can actually agree on how to let industry move forward in a way that's safe.
05:48On that note, I want to play some sound, if we could, from President Trump talking about what
05:52you just described, that race between the U.S. and China.
05:55We're leading China in AI. We're the most sophisticated country in the world. And frankly,
06:01I want to keep it that way, because whoever wins, AI wins. And we can put guardrails, we can do
06:06this
06:07and that. But I think you have a lot of negative forces that are bringing it up that shouldn't be
06:12bringing it up. How salient is that message as you look at kind of the dynamic between
06:16the U.S. and China when it comes to AI development?
06:20That is the key tension here. I think even if, you know, President Trump is concerned about safety,
06:26what he's saying is that he does not want to lose the AI race to China, essentially. And so
06:30having China on board with any kind of safety plan in a really meaningful way with the U.S.,
06:38that would be a major kind of diplomatic win. I think it'll be very interesting to see what comes
06:44of these talks. On the other hand, Trump does allude to some unnamed forces that he thinks are
06:52perpetuating these safety fears in a way that's not constructive. I think that's a big debate right
06:58now. How seriously should we take these concerns that we're hearing from AI companies? A lot of
07:03people have been questioning if this also serves to help them in some way. And it's sort of so bad
07:09that it's good, their technologies. What I will say is that for a long time, people in the industry
07:14have been raising these concerns. And for a long time, many people have been calling them sort of
07:20overly paranoid or overly sort of worried about these hypothetical risks. I do think with cyber,
07:27we have seen some of those risks actually turn into real, you know, damages or unintended kind of
07:35harms of the AI. But there are many fronts where we just still don't know. We don't know how far
07:39away
07:40we really are from, let's say, bio risks and AI or more sophisticated types of potential AI harms.
07:48So that's what is sort of at stake here between President Trump and the Chinese government
07:55leadership. And how seriously should these two countries be taking these safety concerns? And
08:00what can they actually, if anything, both get behind in terms of guardrails?
08:04Let me ask you lastly, kind of what can be done in the face of this existential threat or this
08:09debate
08:10over the existential threat? Yes, there's a lot of like beard stroking and thumb sucking about what
08:14might might happen here. But we did see Governor Gavin Newsom of California sign an executive order
08:19calling for the acceleration of the implementation of safety checks and advancement of the creation of a
08:23kill switch within frontier AI programs. The fact that these labs are directly issuing urgent warnings,
08:28he tweeted, about the risks of AI should be deeply concerning to everyone. What tangibly can be done?
08:34Again, there's debates about policy, debates about regulation. Is the creation of a kill switch like
08:39this a realistic thing? I know there was talk of melting the chips if things were to really go south
08:43in the AI space. Well, I, you know, of course, destroying chips probably is not something that
08:51certain leaders in the industry like Jensen Wang would not be a fan of that, I would imagine,
08:58along with all the investors in that company. But I look, I think that the idea of a kill switch
09:05has
09:05long been discussed in AI circles. Exactly what that would look like is sort of up for debate,
09:10because as you're you're mentioning at the end of the day, if these AIs and this is all hypothetical,
09:16but become so powerful that they can sort of override human instruction on how they should behave on a
09:21software level, then at some point you do have to pull the plug on the hardware, so to speak. Right.
09:25So what does that involve? Is that just stopping the AI from having electricity or do you need to
09:30stop the AI from having the chips at all? You know, there are these hypothetical scenarios that
09:35the AI kind of doomers have laid out of, well, what if the AI takes control of the electrical grid
09:40and
09:40then what? And so it's very easy to kind of go to the darkest scenarios here. I think the key
09:45question will be to your point, what would an actual kill switch look like in effect? And would
09:49that be effective even if the AIs were to misbehave in the most nefarious ways we know possible today?
Comments