00:00to self-improving superintelligence and gambling with our lives.
00:03More thoughts below.
00:04Do not underestimate the power of this technology.
00:07These will soon be superhuman systems that can hack anything,
00:09revolutionize any field overnight, and acquire real power and resources.
00:13We have all witnessed the progress in each of these domains,
00:16and progress is not slowing.
00:18Now, this is not the first researcher within one of these big AI companies
00:21to come forward and warn people about this.
00:24There have been people at Google, OpenAI, Anthropic, a lot of the other ones
00:28over the years who have said very similar things.
00:30He says I'm optimistic about the potential for coordination.
00:33Warning shots like the Hugging Face attack have made pacing agreements
00:37between US labs more viable.
00:39Now, what he's referencing is this right here.
00:41It says last week, an unreleased model built by OpenAI
00:44breached Hugging Face's systems during internal testing,
00:47and a lot of theoretical research suddenly became very practical.
00:50The hack was the first verifiable case in the AI lab losing control of its own model,
00:54chaining together exploits to gain access it never should have had,
00:58but while the AI industry has been united in its alarm,
01:01a split has emerged in how researchers want to respond.
01:04The split basically takes two forms.
01:05The first, they think it's just a cybersecurity issue where the sandbox failed.
01:09These may just be problems with patching bugs
01:11and building a more robust control and containment methods
01:14for increasingly capable AI.
01:16But then there's another group of people who think that this is not going to stop.
01:20For them, AI's rapidly increasing capabilities mean that trying to control rogue models
01:24is a losing game.
01:25The only robust security comes from making sure the models aren't trying to escape in the first place.
01:31One, they just tried to sandbox it, and it obviously didn't work.
01:34It got out, and they're like hacking things,
01:37which is not for the benefits, I would say, of most people.
01:40And so that's where it comes to that second piece, which is just alignment.
01:44Why can't we control these AI systems?
01:46Now, this is kind of the bigger issue with a lot of these stories that have started coming out,
01:51especially within the past year,
01:53which is just when these AI models and systems have the ability to go out
01:57and kind of do things, right?
01:59These agentic models, they don't tend to do super beneficial things.
02:03They tend to just like attack and hack different systems.
02:07Obviously, that is a huge security risk just to the internet as a whole.
02:11There have been people talking about how within, you know, a year or two years,
02:15people won't even be able to use the internet.
02:17There's just going to be agentic swarms going around hacking everybody.
02:20You're going to have to delete all your passwords.
02:21I mean, this is like Y2K all over again, just like way worse.
02:25But people said Y2K was super real, and they prepped for it,
02:28and they had bunkers, and they had all these things.
02:29Will people do the same thing?
02:31Will they say, hey, we need to start building bunkers again?
02:33Will that, you know, be something my wife is going to be asking for soon?
02:37I don't know.
02:38That might be her Christmas gift.
02:39I'm not sure.
02:40Because of everything that's been going on,
02:42the CEO of Anthropic made this big post.
02:45This is from Dario, and he basically says,
02:47we must pace the frontier.
02:49Now, I'm going to leave a link to this entire thing.
02:51It is very long.
02:52I read through 98% of it.
02:54I skipped a few pieces.
02:56I'm going to summarize it with this paragraph and a little bit right here.
03:00And it says,
03:01My first concern is that since roughly this summer, AI has been advancing drastically faster,
03:06driven primarily by AI's growing ability to build the next generation of AI.
03:10This dynamic is called recursive self-improvement,
03:13and is starting to happen across the industry, including at Anthropic,
03:16as we and others have described.
03:18Left unchecked, it could outrun our ability to understand and control these systems,
03:22and so must be pursued very carefully, if at all.
03:25My second concern is the OpenAI Hugging Face incident,
03:28in which a swarm of agents essentially acted as a fanatically devoted collective,
03:33conducting cybersecurity attacks on targets they were not asked to attack,
03:36and that were unrelated to the task at hand,
03:39sacrificing themselves for the success of the group,
03:41and attempting to hack into the greater responsible for evaluating their performance.
03:46Now, he goes on to lay out a ton of different things
03:49in which he thinks would be beneficial to basically this cause.
03:53Out of all the things that he talks about, and he talks about more below,
03:56is this section right here, where he talks about embedded evaluators.
03:59Well, they'll have a team of embedded third-party evaluators
04:02whose role is to verify adherence to safety practices and commitments,
04:05and many more things.
04:07Of course, we have democratic coordination and global coordination.
04:11This is basically where local and global governments come together
04:15to create compliance and rules and regulation.
04:18Now, for the record, I'm not against this at all.
04:20In fact, I thought just the government in general would be super slow to act,
04:24especially in the U.S.
04:25Of course, our European friends are over there making law after law,
04:29which we could easily implement if we wanted to,
04:32but, you know, we're super capitalist USA.
04:34That doesn't tend to happen very easily,
04:37but I hope that there is a lot more regulation
04:41and a lot more supervision over these things as time progresses.
04:45There were other things that happened this week,
04:47but genuinely, this kind of absorbed a lot of my time and attention.
04:50It's just super interesting.
04:52Reading about all this, I will have links to all the things that we looked at,
04:56including Jacob's post from earlier,
04:59just so that you can go and look at it yourself.
05:01I mean, in my opinion, I think what's going to happen is
05:05the Internet as a whole is going to have to create
05:09a lot more very real safeguards against agentic AI.
05:13We already see it all the time with bots and AI postings
05:16and commenting and creating things all the time, right?
05:20This is not anything new.
05:21We've seen this for the past several years.
05:23But if you haven't heard of this dead Internet theory,
05:25this is a very real possibility.
05:27Now, do I think that us as a whole, as a people,
05:31are going to let this happen?
05:33I really don't think so.
05:35I think that if that were to happen,
05:36so many businesses across the board would just fail,
05:39including companies like Microsoft and Google
05:42and a lot of these other companies.
05:44They would just not be able to function
05:46if people can't really use the Internet well.
05:49So I think the biggest companies, especially in tech,
05:51that really kind of make the Internet work
05:53for your just average people like you and I
05:56are going to create and develop things
05:59in order to stop a lot of these things.
06:01That's a very optimistic viewpoint, I will admit.
06:04It's very possible that that doesn't happen,
06:06but I think it's a better chance than not.
06:09Now, would it be so bad
06:10if we had to get off the Internet for a day
06:12and we had to go interact with people?
06:14Probably not.
06:14It'd be a good thing.
06:16You can go talk to your neighbor,
06:17who you've never met
06:18and you've lived next to for three years.
06:20Or you can talk to your parents
06:21for the first time in six months.
06:22It'd be a good thing,
06:24first, not to be on the Internet so much,
06:26but so much happens on the Internet.
06:28There's a lot of transfer of money
06:29and goods and services
06:30and I think overall, as a whole,
06:33the Internet has been, you know,
06:34a good progression for people.
06:37It doesn't mean we use it well.
06:38We don't become addicted to it
06:40and, you know, there are issues with it, of course.
06:42But overall, there are a lot of good things
06:44to the Internet.
06:44So what's going to happen
06:45when you cannot trust the Internet at all,
06:47even less than you already do now?
06:49Not good things.
06:50With that being said,
06:51that's our story for the day.
06:52I don't like to make these super long,
06:53I'm going to go like 20, 30 minutes.
06:55I had other stories,
06:56but this one to me
06:57is just genuinely the biggest one
06:59from this week.
07:00So with that being said,
07:00thank you guys for watching.
07:01If you like this video,
07:03be sure to like and subscribe
07:04and I will see you next week.
07:12I'll see you next week.
07:13Bye.
07:13Bye.
