00:00Anthropic has said that its new Mythos preview is too dangerous for public release.
00:04And Secretary Besant and Chair Powell called in the heads of the major banks to warn them about it.
00:12I applaud Secretary Besant for doing that.
00:15One of the things we always were focused on, the risk to the banking system, the risk for any kind
00:21of cyber attack.
00:22You're just getting in and change the ones and zeros around in the banking system.
00:27And so much of it rests on our banking system. So nothing is more important.
00:31We need to do something about it. I don't think there is a framework.
00:34You know, everybody is keen to have a framework within which to operate.
00:39I don't think there is a governance framework that is there to actually mind those things.
00:44We need we need to work on that.
00:46Catherine Judge is a professor at Columbia Law School specializing in the Fed.
00:50She explained the risks that Mythos could pose for the banking system and what regulators can do to manage them.
00:57The big deal is there are real concerns that this new, much more sophisticated form of AI could detect and
01:05likely will detect very meaningful vulnerabilities in banks software in ways that could expose them to cyber attacks that would
01:13disable individual banks and potentially the broader banking system.
01:16There is the danger of bad actors getting access to it. Is there a flip side to it also?
01:21They may give the banking system and banks some defensive tools, although it might lack.
01:26Yeah. I mean, this is always the interesting question is like AI fighting AI. And what does that look like?
01:31And there's also a question of what that looks like for the structure of the banking system.
01:34Again, the U.S. has always had a very diffuse banking system.
01:38We have lots of small banks. We have regional banks and they provide different types of services and loans to
01:43the real economy.
01:43And so one of the interesting questions, if this becomes absolutely vital to be part of the financial system, who's
01:50going to have access to that type of technology?
01:52The financial system is global. Last time I checked. So it's not just a U.S. problem we have here
01:57potentially.
01:58Not at all. And we've seen both European and Asian regulators speak out and voice significant concerns.
02:03And so one of the things that's so interesting about this is you both need clear public-private coordination.
02:09But there's also the possibility of global coordination to try to understand just what these threats are and then what
02:16can be done realistically to try to contain them.
02:18What exactly can the Federal Reserve do? They have an awful lot of Ph.D. economists. I'm not aware they
02:23have a lot of A.I. engineers.
02:24Yeah. I mean one of the real challenges no bank supervisor is prepared for this. Right. So cyber attacks are
02:30nothing new.
02:31It's a challenge banks have been dealing with for a long time. And supervisors have played a role helping to
02:37make sure that banks are appropriately prepared as part of operational risk
02:40and other concerns. But this is a whole different level. It's jumping up the magnitude of the threat in a
02:47way that no bank regulator or supervisor is prepared to fully address.
02:52And so the question is can they ask good questions. Can they help provide guidance as they did in Washington.
02:58Can they facilitate coordination and conversation. You turn to the Fed right now to avoid the problem.
03:04But if heaven forbid there is a problem there is a serious hack. What could the Fed's role be in
03:11remediation.
03:12This is one of the key issues. I mean the government's not going to be able to stop a serious
03:17cyber attack.
03:18On the other hand the government can play a very meaningful role helping to contain the damage. And really when
03:23you're worried about the function in the financial system a lot of it's on prevention.
03:27But things are going to happen and things are going to wrong. And the effort then is on containment.
03:32And that's where the Fed having strong relationships with the entire banking system and helping to make sure that it's
03:38providing liquidity and potentially helping to provide alternatives
03:41for affected consumers. And really coming up with a game plan over what can we do and what should we
03:47do if things go wrong is going to be I think part of the effort going forward to try to
03:52make sure that in the
03:54really unfortunate event that there is a wide scale malicious attack that the damage isn't nearly as crippling as the
04:02actor might hope.
04:03Like the rest of us the Fed may be struggling to understand the benefits and risks of AI. So we
04:09turn to the experts. People like Margaret Mitchell researcher and chief ethics scientist at Hugging Face an open source AI
04:17platform mythos is really on the cutting edge of cyber security right now. It's incorporating a lot of things that
04:23we've already known that's already common in cyber security. But it's within a system also called mythos that can operate.
04:32Warp speed. So there's this new sort of paradigm right now within AI called AI agents and agentic computing. Mythos
04:41is an autonomous agentic system. It can take sequences of actions without any human oversight. And it can do it,
04:50you know, while you're asleep and infiltrate tons of different systems.
04:53If I'm one of the big banks, the Secretary Treasury Besson called in and I learned about this, what can
05:01I do about that? The big banks have been spending a lot of money on cyber security for quite a
05:05while now.
05:06Right. I think that that knowledge and that expertise is really important here.
05:11You don't necessarily need mythos to build up appropriate defenses. And actually this is where the open source world can
05:19really help because it's quite possible or at least we sort of think it's possible within the world of AI
05:25to have different models swapped in in a similar system built with deep security expertise to help with probing for
05:34vulnerabilities and creating patches that kind of thing.
05:36And so I think it's really important right now for people to recognize a that this technology is possible and
05:43that doesn't have to be siloed to any one company.
05:47And B, that you can build up defenses right now using tools that are already available for intruder detection, fuzzing
05:55frameworks, these kinds of things, where basically it's possible to preempt any sort of attackers and defensively set up safeguards
06:05against any malicious problems.
06:07Part of cybersecurity defense is basically having your own organization or other organizations try to hack into you to see
06:15where there might be vulnerabilities.
06:16So that's been going on for some time at the banks. But is this different in the sense that it
06:21can be running in the background constantly?
06:23The punchline is that Mythos is operating in a way where human oversight can be difficult. And that's because it
06:33can create action sequences without any sort of human intervention and act on different kinds of things that it sees
06:41as possible.
06:42And it can do it very fast and it can spawn what are called sub agents, so little workers that
06:48can do the same thing across distributed systems. This is all this sort of new paradigm of agentic computing.
06:55So it's really within this calibration of autonomy that I think we have to think critically about what we want
07:02these systems to be doing, where we want to bring in human oversight, how people should be able to interact
07:09with these systems in order to approve or deny things.
07:12Because otherwise, with full autonomy, they can just keep going overnight while you're sleeping, spending tons of different systems, that
07:21sort of thing.
07:21So, you know, you have to put in place those guardrails, those checks in order to make sure that humans
07:27are in the loop and can see into the loop, you know, can really understand what's happening.
07:32So you, of course, are an expert in this. Give us your best judgment when you talk about fully autonomous
07:37versus semi-autonomous. Which is better if the goal is safety?
07:43Semi-autonomous. I've actually written papers on this and op-eds and I'm very opinionated about this, actually.
07:48So when you have full autonomy, that's super flexible, you can do lots of different things and so that's pretty
07:54cool.
07:54Lots of different things you'd never thought of doing, you don't have to pre-program it, right?
07:59But from a safety perspective, also within the context of the financial sector where there's a lot of regulations at
08:06play in terms of accountability, in terms of the passage of data, that kind of thing.
08:10You want to make sure that you're able to specify specifically the context within which the system can run and
08:16that it can't jump out of those contexts.
08:19If you're a regulator, what do you do in the face of this new technology to try to minimize or
08:24prevent systemic risk?
08:25Regulators have one piece to play within the overall accountability chain.
08:32And I think that for regulation that focuses more on trying to help ensure reliability, ensure safety, opening up the
08:40possibility of third party audits, third party testing.
08:44So there's something to be said for companies that share their code, their information, even if they're proprietary, in order
08:52to undergo third party auditing, third party testing, that kind of thing.
08:57Regulators are in a really nice position to try and ensure that people's rights are protected.
09:03So what are the rights that should be protected?
09:06So this is where security comes in, reliability, safety, privacy, those kinds of things.
09:12So I think regulators have a role to play within the overall ecosystem in setting up what kinds of human
09:19values should be protected in these systems.
09:22And how can you ensure that your system does that?
09:25Show us, run the evaluations, have it undergo third party evaluations where we can actually see from someone without the
09:33phone,
09:33the financial incentives of one institution, how well these systems actually work.
Comments