- il y a 2 semaines
Building at the Frontier
Catégorie
🤖
TechnologieTranscription
00:00Raise your hands if you've ever heard about the best-selling book and movie, Too Big To Fail.
00:10Too Big To Fail. Never heard of it? Raise your hands!
00:16Okay, so a lot of you have heard of it. What about the drama series Billions?
00:23Okay, some of you are raising your hands there.
00:26Well, guess what? The author and producer of Billions and of Too Big To Fail is here to lead the
00:34next conversation.
00:36We are talking about a prominent American journalist in the finance world.
00:41As we challenge the evolving landscape of artificial intelligence, we want to hear from leaders who ensure AI aligns with
00:50human values.
00:52So joining this conversation is the co-founder of Anthropic.
00:57He previously worked at OpenAI, but left to co-launch Anthropic in 2021, aiming for a safer approach to AI.
01:09And actually, something interesting here is that he used to work at OpenAI with his sister, and then they both
01:14left to launch Anthropic.
01:15They were both named and included among the times 100 most influential people in AI.
01:23Here to build at the frontier, please welcome Dario Amadei, CEO and co-founder of Anthropic.
01:33In conversation with Andrew Ross Sorkin, anchor at CNBC, author and New York Times bestseller.
01:42Good afternoon, everybody. Thank you so very much for joining us.
01:46I am thrilled to be here with Dario and to talk about the future of AI and where it's all
01:51going, where investments should be made, and perhaps where they shouldn't.
01:56But here's where I want to start with you this morning, as we're all trying to understand the future of
02:01artificial intelligence, given you've been such a pioneer in this.
02:04And I should say, a pioneer worked, by the way, at Google and Baidu and OpenAI before this, and has
02:10a background in neuroscience and biophysics, which really parallels this entire universe.
02:16And my understanding is, when you were a toddler, is this true, that you used to carry around a calculator
02:21with you?
02:22I don't know how you know that, but that's true.
02:24Okay, so there's some math going on, but where I want to start this conversation is actually an announcement that
02:31you made yesterday.
02:33Some research, really breakthrough research, that really helps us get inside and understand what I think were black boxes, largely,
02:41around AI.
02:42And maybe you can just tell everybody about the research and how important it really is, because I think it's
02:47such a shift, potentially, in the future of this very issue.
02:51Yes, so as you mentioned, today's modern AI systems are made from these large neural nets that have hundreds of
03:00billions of parameters, hundreds of millions of neurons, and it's very hard to understand what's going on inside them.
03:07So, since almost the founding of Anthropic, one of our co-founders, Chris Ola, helped to invent the field of
03:15what's called mechanistic interpretability, which is trying to look inside the mathematical behavior of the neural nets and understand what's
03:22going on inside them.
03:23And so we've had a research program on this running for the last three and a half years.
03:27It was pure research up until now, and so the research that we put out yesterday was we showed in
03:35one of our production-grade models, Claude III Sonnet, which is the middle of our three models.
03:41We're able to look inside the model and find 30 million what we call features, which are things that correspond
03:48to concepts, and we can watch them light up as the model reads content and generates content.
03:53So an example of this is there's a feature that corresponds to when an actor breaks the fourth wall and
04:01says something that makes them, you know, that shows that they're aware that they're in a fictional setting.
04:05And if you ask Claude, our AI model, about itself, that feature will light up as it starts to talk
04:12about itself as an AI system.
04:13And we can turn on and off that feature, and when we turn on and off that feature, Claude's behavior
04:19changes.
04:19There are features for security vulnerabilities in code.
04:23There are features for bias for or against various kinds of groups.
04:28There are features that correspond to different genres of music.
04:32So we're starting to see a whole atlas of what's going on inside these models, and we're able to intervene
04:37into them and change their behavior, which is never possible before.
04:40I was fascinated by this idea of sycophancy, this idea that you can actually see the model being a sycophant.
04:49Yes, there is a sycophant feature.
04:51So when you turn this feature up, the model will give just completely over-the-top praise.
04:58Like, if you ask it, like, how are you?
05:00It'll say, like, you are so thoughtful for, like, asking me how I am.
05:05Like, you're the greatest person in the world.
05:07You know, just crazy over-the-top praise.
05:11So this causality, this ability to intervene, you know, that's the ultimate test of whether you understand systems.
05:17And we have a long ways to go, right?
05:19But this is showing the beginnings of something amazing.
05:22Do you think we understand when and how these models hallucinate?
05:27We're starting to understand that.
05:29There also were features for deception.
05:33There are features for incorrect facts.
05:37Various things like that.
05:39Now, when models hallucinate, they don't always know that what they're saying is incorrect.
05:43Just as, you know, humans make a distinction between lying and simply being misinformed.
05:49So I don't think this is the only source, but reducing hallucinations is, I think, one important application of this
05:56discovery and hopefully other discoveries that will follow it.
06:00In terms of where this research can take you and whether it'll be applicable to other models,
06:06do you see OpenAI or Google or anyone else be able to use what you've now learned so we all
06:13can actually understand what's happening?
06:15And by the way, does that become a trade secret unto itself?
06:18What's really happening inside these models?
06:20Yeah, so I think we want to go in two directions, right?
06:22I think to a great extent, the basic science of understanding what is going on inside the models, we think
06:30of that as a public good.
06:31Anthropic is a public benefit corporation.
06:32And so we want to make the basic principles available to everyone.
06:37We think that's important for making models safer both in the short run and in the long run.
06:42At the same time, I think we are interested in using these insights to either improve the safety or performance
06:51of our models or to offer commercial tools that would enable customers to understand what's going on inside the model.
07:01And so I think we're probably not going to publish every detail, but we think the overall basic science has
07:12an important...
07:12Colleagues and peers in the business have gone to places like Washington and Brussels and elsewhere to say,
07:19we're okay with regulation, we want regulation.
07:22Do you think that you can regulate yourselves?
07:27Yeah, I mean, so, you know, I'd go in a couple directions with that.
07:31One is, you know, I think at the end of the day, like, you know, we can be helpful and
07:36we can provide perspective, but there should also be independent perspective, right?
07:40You know, there are academics who think about issues of AI safety, and, you know, of course, our company or
07:46any company has its own incentives.
07:49And so, you know, if I was a regulator, if I was a lawmaker or policymaker, you know, I would
07:56listen to what the companies have to say because they have certain expertise,
07:59but I would also listen to what academics and independent researchers have to say.
08:04Let me ask you about this.
08:05I don't know if you saw, I'm sure you followed it, over the last week and a half, OpenAI, which
08:08was your former employer before this,
08:10had a number of major executives leave, including someone who ran, effectively, the entire safety operation.
08:17But one of them went to Twitter and said, safety culture and processes have taken a backseat to shiny products.
08:24I'm curious, when we, as the public, see people of that stature go out publicly and say these things, what
08:31someone like you thinks of that?
08:32Yeah, I mean, I can't obviously speak to what's going on at a company I don't work at, but, you
08:39know, I can talk about our priorities.
08:42And, you know, I think it's important to do both.
08:45I think it's important to always take a look in the mirror and make sure that we're doing a good
08:50job of balancing the two.
08:52I think, you know, our thesis has always been that a lot of the key safety work requires models to
08:58be scaled.
08:59Scaling models is expensive.
09:01And so, you know, it requires some funding mechanism.
09:05And that means, for at least, you know, for at least a lot of the safety work that needs to
09:09be done, you know, it may need to happen at companies.
09:12And the interpretability work we talked about is a great example of that, right?
09:15You can do it on very small models, but there was so much challenge to really, you know, showing that
09:22it scales.
09:23And it only could have been done at a scaled business.
09:25But on the other hand, we need to make sure that, you know, we don't lose our interest in safety,
09:33right?
09:34That's why we make these interpretability work available to everyone.
09:38One of the kind of features we have that helps us to balance these things is this thing we have
09:44called a responsible scaling plan.
09:46And what the responsible scaling plan does is it has certain thresholds for when, you know, we see certain concerning
09:54behaviors in models,
09:55many of which have not yet been observed in any models and we expect in the future.
09:59Things like misuse of biology or, you know, cyber attack or cyber defense.
10:06And the point of the RSP is that in order to deploy our models once certain thresholds are hit, we
10:13have to take certain measures to comply with the RSP.
10:16And a lot of those measures require us to do things like understand what's going on in the model better,
10:22be able to steer it.
10:23And so it's a way of putting a constraint on ourselves where safety becomes a product requirement.
10:30It goes into the org roadmap, even as the org grows, right?
10:34As orgs grow, you know, revenue, products, really easy to understand, right?
10:39And so if you don't push back against that, that's what you'll get.
10:43But the RSP gives us a way to insert safety into our product planning, into the everyday business of the
10:51company.
10:51Okay, let me ask you about another big headline because it has to do with intellectual property.
10:55And there's a lot of creative people here in this audience.
10:58The Cannes Film Festival is taking place just a bit from here as well, down in Cannes.
11:05You probably saw the news about the Scarlett Johansson voice or the Scarlett Johansson sounding-like voice.
11:13It was effectively voiced by another actress.
11:16I know this was an open AI platform issue, and I know it's not yours.
11:21But I'm curious how you think about the model and whether you think the model should be allowed to do
11:28something that sounds like this.
11:30You know, I've prompted, and I'm sure other people have prompted your service and others to say,
11:36write me something, write me a poem that sounds like this poet.
11:42Or write me a song that sounds like this artist.
11:46Should they be able to do that?
11:48Yeah, so I think one of the reasons that Anthropic has focused more on text than on multimodal output modalities,
11:56especially such as images or video, is that, you know, those modalities bring some complexities.
12:03That, you know, we're a small company, and so, you know, we want to be focused,
12:07and, you know, we prefer to focus on things that have less of these issues.
12:13Not to say that we'll never do these things in the future, but we want to think carefully about how
12:18we would do them.
12:19Now, I think even within text, as you've alluded to, some of these issues come up.
12:24And so I think this is, you know, this is something that we should be thoughtful about in terms of,
12:29you know,
12:29thinking about where data sources come from, you know, what outputs are okay.
12:35I mean, one thing we've been very clear on, I think everyone agrees, like, you can't output verbatim copyrighted content.
12:42Right.
12:42Right?
12:42And I think everyone agrees on that.
12:44But here's where I think it gets more complicated.
12:46If I said I wanted to write a book in the style of James Patterson, right, I could technically try
12:54to write a book in the style of James Patterson.
12:56I might do well.
12:56I might not do well.
12:58But should an AI system be allowed to write a book in the style of?
13:03Should it be allowed to write a song in the style of Taylor Swift?
13:08Should it be allowed to write a film that is in the style of my namesake, Aaron Sorkin?
13:15Again, I mean, you know, it kind of comes back to have you trained on the data that kind of
13:21enables you to do that, right?
13:22And these things are a little bit fuzzy, right?
13:24Because you can, you know, even if the model hasn't read content by a particular person, right, we're kind of
13:31building general cognitive tools, right?
13:33So I think in the long run, we're really going to need to grapple with the idea that as these
13:39AI systems become very smart and capable,
13:42they're going to be capable via all kinds of routes of kind of doing these feats that infringe uncomfortably on
13:51what humans are able to do,
13:53not even specific humans.
13:55And so, you know, we're going to need to think about as a society what our rules and thinking about
14:03this are, right?
14:04I want us to think big in terms of how society is organized because AI is a very broad technology.
14:11Okay, here's a big business question, and I ask it on behalf of every venture capitalist and startup in the
14:15room
14:15who's thinking about how they're going to integrate AI or build a business on either the back of AI or
14:21in the world of AI.
14:23If you were successful, a lot of these large language models, as they develop and get better,
14:29are going to ultimately undo any kind of moat that exists around a lot of these businesses and a lot
14:35of apps.
14:36There's a lot of apps being built right now, even on top of large language models.
14:40If you weren't the CEO of Anthropic and you said, I want to be in the AI business today,
14:46what businesses would you go into and what businesses would you stay so far away from
14:51because you know that ultimately these models will ultimately be able to already do that?
14:56I would think about areas that connect AI to some other very separate type of expertise.
15:04So, you know, I previously was a computational biologist, computational neuroscientist,
15:10so probably I would start a company or I would invest in a company that applies AI
15:15to, you know, some area of biology or health.
15:19And the idea is that, you know, I take the models, that's some new ingredient,
15:24and I try to understand them and I combine them with this other expertise that I have.
15:29I think the same sorts of comments could be offered for areas like robotics,
15:35for areas like energy technology.
15:37What I would not build is something that's a thin wrapper around the technology.
15:41I would think about what is the expertise that this other company brings to the table
15:48that they're able to combine with AI that's provided by companies like Anthropic.
15:52And you don't worry that the expertise is ultimately going to be learned by AI?
15:58I mean, I think that's the fundamental question,
16:00trying to understand what expertise the AI will ultimately be able to learn itself
16:05and what expertise it will never be able to learn.
16:08Yeah, so I would again separate it into the long-term and the short-term, right?
16:12In the short-term, I'm confident that areas like biology, areas like robotics,
16:20there's just a huge amount of human expertise and know-how that, you know,
16:25today's language models may know something about the field,
16:28but, you know, being able to actually slot it in,
16:32there's going to be a lot of human expertise and human structures that are required.
16:36I'm very confident of that in the short run.
16:38I think in the long run, again, it ties into this bigger question of,
16:42will there be a time when AI systems are better than most humans at most things
16:48or even better than all humans at all things?
16:51And I think once that happens, we're all in the same boat, right?
16:54We're all in the same position.
16:56You know, like whether you're a manual laborer or the CEO of a company
17:00or, you know, or an actor or actress, it's the same thing.
17:04We're all going to be in the same boat.
17:06And so we're going to need to rethink as a society, you know,
17:09how we deal with that and how that works.
17:10So while we're on that topic, and then I want to move to the sort of
17:13closed versus open model question,
17:16in terms of jobs, labor, 10 years from now, 5 years from now,
17:22what does it look like in your mind?
17:24Yes.
17:25So I would, again, separate into this long-term and short-term.
17:28I think in the short-term, we have this strong feeling that AI will be
17:33complementary to humans.
17:34So there's this economics researcher, I think you know him, Eric Brynjolfsson,
17:39who's done these studies that show that by default, folks who try to apply AI,
17:45like downstream customers, will often apply them in ways that, by default,
17:50replace humans.
17:51But if they're more thoughtful about it, if they find a way for AI to complement
17:56humans, that ends up increasing productivity more.
17:59So we very much want to encourage these complementary applications.
18:03They're both good for jobs and for humans, and they end up being more productive.
18:09That's the short-term prognostication.
18:11That's the short-term.
18:11Okay, now go long-term.
18:12Again, again, the long-term.
18:14If we reach this world where AI systems are as good as most or all humans at most or all
18:22things, you know, then we don't need to think about that, right?
18:26Like, we've had a long era, the industrial era, during which people derive their value from
18:34what they're able to produce economically.
18:36And I think in the long run, on longer time scales, AI may challenge that assumption, just
18:41as the industrial revolution, you know, challenged the social structure of feudalism and farming
18:47challenged the, you know, social structure of hunter-gatherer tribes.
18:50And so I don't know what the next thing in that sequence is, but I think we do need
18:54to start thinking about what it is.
18:55We have a lot of young people in this room, but we have some parents in this room.
18:58I've got three kids.
18:59If you had kids these days who were going off to college, what would you tell them to learn?
19:06Well, you know, I would definitely tell them to get familiar with AI.
19:10But I think a broader, you know, lesson, particularly with a lot of the, you know, you mentioned
19:16some of the synthetic content being created by AI, I think teaching people to think critically
19:23about the information ecosystem is going to be one of the most important things, right?
19:27I think AI can maybe help us.
19:30We may have AI helpers that help us to navigate the information ecosystem.
19:34But at the end of the day, AI can help just as AI can hurt, but it has to come
19:40from you, right?
19:40You have to have some basic skepticism, some basic critical thinking faculty in order to
19:48process and understand the world and not fall for the many people that are trying to...
19:52And what am I supposed to do if my kids go on and prompt Claude your system to help them
19:57do their homework?
19:57Am I supposed to like that or hate that?
19:59Yeah.
20:00I mean, it depends how they're prompting it to do their homework, right?
20:02If it's, you know, if it's just, you know, end-to-end, you know, doing the tasks they're
20:09supposed to do, then, you know, either they're doing something they shouldn't do or maybe the
20:13task that was being designed was not.
20:13Do you think that there's going to be tools very soon on services like yours to prevent
20:18kids from writing their...
20:20Having these services write their papers for them?
20:22Yeah, I think those will exist in some form.
20:24But maybe what I'm even more excited about is actually thinking from first principles about
20:29how these models can help with education instead of short-circuiting education.
20:34So this thing I mentioned before about, you know, take that where I see the real value
20:41is AI plus some other skill.
20:44There are people who've thought about education for a long time.
20:46I would like some of those people to, you know, to work with companies like us and, you
20:52know, there are things we're thinking about in this space and try and design how things
20:56will work.
20:57One thing I've often, I've often cited, you know, when I was a kid, I read this book
21:02called The Diamond Age, right?
21:04And included in it is this thing called The Young Lady's Illustrated Primer, which today
21:08we would, you know, today we would call basically an AI system that, you know, that helps with
21:14your education.
21:15And so can we make some version of that instead of like, oh, you know, I went to, you know,
21:19I went to this AI model and like, you know, it wrote up my essay for me.
21:22Like that seems dysfunctional.
21:24I mentioned earlier we're going to talk about open models versus closed models.
21:28You have a closed model.
21:30Open AI has a closed model.
21:31Google has a closed model.
21:33Meta just came out with Llama 3, which is an open model.
21:36And a lot of people think it's a pretty remarkable model and is on pace to get close to what
21:41some
21:41of the closed models are able to do.
21:43What do you think of both the opportunities and also the challenges that the open models
21:49create, the benefits and some of the challenges, but also what does it do to the business, which
21:56is to say that if there is an open model out there that's as good as a closed model, what
22:01does that do to the economics of the closed model business?
22:04Yeah.
22:04So, again, I would talk short-term, long-term.
22:07So, I think, you know, in the short-term, in terms of safety and security concerns, I'm
22:11not concerned about any of today's open models.
22:15I'm not concerned about, you know, open models that come out in the next year.
22:18I'm not concerned about closed models that come out in the next year either.
22:22But, you know, I think in the long term, you know, we have various safety and security concerns
22:28about models in the long term.
22:30But I think this is more about powerful models and future models versus today's models than
22:36it is open versus closed models.
22:39And, in fact, I think open and closed models are becoming more like each other.
22:43So, you know, on one hand, I think we've seen this pattern where when companies or efforts
22:48are getting started, they'll often do open weights models and they switch to closed source
22:53models as they gain traction, as they have products.
22:57At the same time, closed source models are increasingly offering things like fine-tuning,
23:03customization, many of the benefits that you get from open source models.
23:06For example, we're planning to offer such benefits through Amazon AWS.
23:13And, similarly, clouds are starting to increasingly host open weights models.
23:18You have to pay for the cloud service.
23:20There may be some fee that's charged on top of it.
23:22So, in many ways, they're converging to things that are very similar to each other.
23:27We've got to wrap up.
23:28But I was going to ask you, Elon Musk is going to be speaking here tomorrow.
23:31He is building both the vehicles, Tesla, using AI, increasingly using Vision to try to do this.
23:39And, of course, he's using Grok on X, which formerly Twitter and the like.
23:44When you think about the future of AI, you're working on text for the most part right now,
23:49though, increasingly, I think some images that you can input.
23:51We have images in, yeah.
23:53Who has the, when you look at who has the great advantages in the future about data,
24:00some people say Google will have a great advantage because of all the data.
24:04Does the winner come from data?
24:05Does it come from processing power?
24:07How do you stack rank everybody that's out there now and what the opportunities and challenges are?
24:14So, I think many kinds of data are overrated because, increasingly, not just us,
24:19but other AI companies are starting to get good at creating synthetic data.
24:24And so, I think data will generally not be a bottleneck.
24:27For specific applications, particular types of data will be important, right?
24:31But isn't synthetic data very dangerous because that's just basically creating robotic data
24:37based on previous data?
24:40Yeah, you can't, the ratio of synthetic to real can't be infinite.
24:44But in terms of the models we want to scale for the foreseeable future,
24:49it's hard to say for sure, but synthetic data is looking very promising.
24:54I think the thing that will determine who has the best model is talent,
24:59invention of the best architectures, use of the data and compute that we have,
25:04as well as we can possibly use that data and compute.
25:07I think in any industry, it almost always comes down to talent.
25:10Dario, thank you so very, very much for the conversation.
25:12Thank you for having me.
25:13Appreciate it.
25:13Thank you, everybody.
25:16Thank you.
Commentaires