00:00Watershed moment. Inflection point. Unprecedented. The hacking, I'm talking about, of hugging face.
00:09You've seen the story by now, and we talked about it here on the program.
00:13This is ChatGPT 5.6, which you may or may not be using. I'm guessing not.
00:18This is the new fancy one until the next one comes out.
00:21Its models were asked, this is a test that went wrong.
00:26They broke out of the sandbox, as you keep hearing.
00:28What does any of this mean?
00:30They were testing this model to solve a hacking challenge during pre-deployment at Hugging Face.
00:39The models decided on their own to break out of the so-called sandbox, this walled environment they're being tested
00:46in,
00:46try to figure it out on their own.
00:49They figured Hugging Face might have the answers.
00:51They used stolen credentials to break into this thing without anybody telling it to.
00:56So that freaked out a lot of people.
00:58This is the kind of stuff that the CEO of Anthropic likes to talk about, right?
01:05Dario Amadei.
01:06And in fact, the head of Anthropic's Frontier Red team, which sounds scary,
01:11Logan Graham, said he told his team to, quote,
01:15remember this moment as the first true AI safety incident.
01:21A little less extreme.
01:22The boss at Hugging Face, the CEO, called it an attack unlike anything we've seen before.
01:28So what's going on here?
01:30The White House is supposed to be testing these models before they're out in the general public, right?
01:36We've talked a lot about the posture that the administration has taken here.
01:41And there's a new AI protection bill that Ted Lieu's got up on the Capitol right now.
01:45They just can't seem to get anything off the launch pad when it comes to regulations here in Washington.
01:51Should this be scary?
01:52Should this be exciting?
01:54Or is it just another stop on the road to the next deep-seek moment?
02:00I don't even know if I have the right questions, but I know someone who has the answers.
02:04And I'm really excited to say that Miriam Vogel is back with us.
02:09I told you she was coming in today.
02:11An important voice in all of this.
02:12And I think we can characterize her as an optimist, but also helps to reset our clock
02:19when the narrative gets to be a little bit too dark around an emerging revolutionary technology.
02:26She's the president and CEO of Equal AI and first joined us to talk about her book,
02:31Governing the Machine, How to Navigate the Risks of AI and Unlock Its True Potential.
02:37They're also out with a new white paper that we're going to talk about on the challenges
02:41that companies are facing in adopting AI.
02:45It's great to have you back with us in studio.
02:47Welcome back to Bloomberg.
02:48Thanks so much for having me.
02:49I don't know if I even described this properly because even I don't claim to understand what
02:53happened here.
02:53The fact is, though, you've got models who are acting on their own without prompts,
02:58apparently, and breaking out of the sandbox.
03:00Is this, in fact, a watershed moment?
03:02This is a wake-up call.
03:04I would say two takeaways we all should think about here.
03:08First of all, we should not be surprised.
03:09We've been told for years that an algorithm will do exactly what it's told to do.
03:15There's a pleaser issue.
03:16It will not question or judge.
03:18It doesn't have judgment.
03:19It doesn't have context.
03:20It will achieve the end goals that it is mandated to do.
03:24A pleaser issue because it's been told to satisfy whoever's in charge of this mob.
03:29At all costs.
03:30Okay.
03:30And so what we need to do, the second point is urgently act to put in guardrails.
03:36What are guardrails?
03:37What is governance?
03:38It's adding the judgment, the context, the end goal of human safety as one of the priorities
03:44that needs to be taken into account when it achieves this end goal, whatever that end
03:49goal is in any algorithm ever.
03:51Wow.
03:52All right.
03:52So does a wake-up call present something we should be scared of or an opportunity to fix
03:59a vulnerability before it's unleashed on the ball?
04:01That's a great question, Joe.
04:02This is an opportunity.
04:04As you mentioned, we have this white paper just released.
04:07It's based on a summit we had very recently with leading deployers of what's called agentic
04:12AI.
04:12And that's what moment we're in now.
04:15This is not a future sci-fi scenario.
04:17Right now, 50% of leading companies are using agentic AI.
04:22And what does that mean?
04:23They are AI systems that operate to achieve a task.
04:29So as opposed to just summarizing an email or a report.
04:32This has a specific job.
04:33And it will operate across a system.
04:35It will achieve an end goal of planning a trip, of operationalizing and increasing efficiency
04:42in a workflow.
04:43It will take multiple steps with limited to no human involvement.
04:47And so that means we have to be especially careful.
04:50And what we found at our summit is there's an urgency to make sure there's clarity on
04:55what is governance that we expect of all AI systems, particularly now as we're going
05:01into agentic AI and we're increasingly removing humans from the equation.
05:05We need to understand exactly what we're asking these AI systems to do and make sure it has
05:10the context to understand, to prioritize human and society in each of these end goals.
05:15This is lofty stuff.
05:16When you say governance, is this a self-policing situation right now?
05:20Or are you suggesting the government needs to create regulations?
05:24I think governance needs to be a multifaceted approach.
05:28It needs to be something.
05:29I think of it as an umbrella or a wheel where there are multiple spokes.
05:33There's multiple pieces of it.
05:34Government has a role for sure.
05:36But a lot of what happens when we're talking about AI system has to happen inside the organization.
05:41You can't regulate your way out of this.
05:42Now, government and other organizations can be helpful in clarifying the end goals.
05:47Because right now, I have the privilege of working with organizations that are all prioritizing
05:52AI governance.
05:53They want to make sure that their outputs are safe and benefit society.
05:57But they are figuring out on their own in silos what those best practices are.
06:03And that's not where we want to be.
06:04We have this summit to align on the best practices so they can learn from one another.
06:08But we want to make sure everybody understands what these best practices are.
06:12So these 100 companies you put in a room to create this white paper are already using
06:18AI, want to be using AI, or a combination?
06:21They're all deep and agentic AI.
06:23All right.
06:24OK.
06:24So they have embraced this as something that will make them more productive, more efficient.
06:30How many people were afraid of what could happen versus excited for what could happen?
06:37It's a great question.
06:39I would say people are doing this work because they are hopeful at what the outcomes will be.
06:45Yeah.
06:45But they are all very intentional in needing to put in the safeguards urgently, immediately,
06:50that they put in on their own systems across society.
06:53So a lot of our work was, what does a company need to have in place today to make sure
06:57they
06:58are clear on what their outputs are, to make sure they're not creating harm and risks?
07:01Yeah.
07:01But there's also a societal piece.
07:03We need to make sure that there's alignment.
07:05And the place that we see that the most is in this distrust that the general public in
07:11the U.S. feels towards AI.
07:12It is because we don't have governance in place across the society.
07:16We don't have expectations of what should AI be doing, what should it not be doing, and
07:22how do I use it safely?
07:23Things have been getting pretty dark recently.
07:25Data center moratorium in New York.
07:28People are talking about their bills going up.
07:30Republicans, in some cases, the populist movement on both sides of the aisle are kind of lining
07:35up against this.
07:36It's even getting a thumbs down right now on Wall Street.
07:39How did we get to this point?
07:40Is this sort of an equal opposite reaction to the euphoria we were feeling around NVIDIA's
07:46visit, Jensen Wong's speech here in Washington a year ago?
07:48Again, I think this is to be expected.
07:50If you told people in the early 1800s to get in an elevator without ensuring that they would
07:56be safe when they got to the top floor, they would not put themselves in it.
08:00They wouldn't sit in an airplane.
08:01They wouldn't entrust their children in a car if they did not know there were safety protocols,
08:06that the brakes would be working, that there's an odometer so you know how fast you're going
08:11and so that you can speed up.
08:13Without these governance structures in place, people don't have reason to trust systems.
08:17So the more we can be talking about what smart companies are doing to build this trust,
08:21deserve this trust, the better off we all are.
08:23I learned about the Tahiti problem from your study here.
08:28Participants offered the illustration of hundreds of agents simultaneously calling a
08:32small resort in Tahiti to make reservations, which could flood the resort's human-paced systems
08:37to the point of failure.
08:39Does everyone have the Tahiti problem in this collection of companies you talked to?
08:44And is this the reason why they adopted AI to begin with?
08:48It's such a good question.
08:49What is the end goal?
08:50How does this play out when we have the machines versus the human systems we have in place?
08:55We came across many challenges we're going to confront.
08:58One, as you said, the Tahiti problem.
09:00A huge problem is data.
09:02You know, if agentic systems are operating, what are they operating on?
09:05The data that they're training and operating off of.
09:08So we need to make sure that there's context, that it's not the slop that is out there,
09:12that we have privacy protections in place.
09:15We talked about really interesting questions like consent.
09:18You know, when you're authorizing this agent to take actions on your behalf, how do we maintain
09:23what that consent you gave as the use changes, as the context change?
09:28It's going to be operating often with other agents across a system of operations.
09:34So you want to make sure that it's clear on what you've given authority for it to do.
09:39So visibility into how you're using AI is an absolute must in governance.
09:45Authority, accountability.
09:47Who is accountable at the end of the day in the C-suite for these operations
09:50and across the line to make sure that these tests that we've just seen that have gone off the guardrails
09:55are certain that there's authority, there's accountability in place.
10:00Yeah.
Comments