- il y a 1 jour
Operationalizing Responsible GenAI
Catégorie
🤖
TechnologieTranscription
00:00Hi everyone, thank you for making it here. It's so early. So I'm very glad to have this conversation about
00:11AI today, which is, like you will see during the day, it's the main topic at Vivatech this year.
00:18And I'm very glad to have around this table three important speakers for that. We have an infrastructure provider and
00:28developer of AI services, a large language model, and end user, if we can say that.
00:35So I'm glad to have Clara Shi. Hi. You're the CEO of Salesforce AI. Thank you for being here. And
00:42then we have Swami. Hi, your name is so difficult.
00:45Swami is just fine. Thank you, Swami, VP AI and Data at AWS. And Michael Selito, you head of Global
00:55Public Affairs at Anthropik. And you just arrived in Europe recently, Anthropik.
01:02So we're going to talk a bit about responsible AI, because that's a thing we don't talk so much about.
01:10So I just wanted you to know, just my first question is very, very simple. But what does, you know,
01:16responsible AI mean to you? Maybe, Clara?
01:21Sure. So responsible AI is AI that benefits humanity, which includes both individuals, businesses, organizations. And for me and for
01:34our Salesforce team, it's about data security, data privacy, ethical guardrails, and alignment with what's good for humans and humanity.
01:43Swani? Yeah, for me, responsible AI is all about promoting safe and responsible development of AI applications, where it is
01:55integrated in the end-to-end development process, so that you build trustworthy and safe AI systems.
02:04I mean, I can share more examples when we get to it, but I hope that kind of gives context.
02:11Yes, of course. And Michael?
02:12Yeah, and thanks for having me here today. I totally agree with Swami that responsibly AI means that you're really
02:17doing things across the entire life cycle, from development, deployments, and even post-deployment monitoring and mitigation.
02:25And anthropic, we consider sort of a range of potential risks and harms from near-term, like misinformation, bias, toxicity,
02:34to longer-term concerns around catastrophic implications of technology.
02:38And the way that we operationalize this is really by doing testing and evaluation across that life cycle, because ultimately,
02:44you can't really measure, you can't mitigate what you can't measure.
02:48And so, a substantial fraction of anthropic just spends all day building and running evaluations across this whole life cycle.
02:55It's really part of the fabric of the company.
02:57Thank you. And as Swami just mentioned, if you can share maybe one story, one use case, one example where,
03:05you know, this approach to responsible AI has transformed an industry or a specific sector. Clara?
03:13Sure. Well, it really does start with trust. And it's amazing. Salesforce, when the company started, before my time there,
03:23trust has been the company's number one value.
03:27And when I look at what we're doing from an AI standpoint, it starts with what we call our Einstein
03:32trust layer.
03:33And so, it's everything from data security, data privacy, data masking, grounding to reduce hallucinations, zero retention prompts to ensure
03:45that our customers' data is never learned by any large language model,
03:50whether it's one of our own or part of our ecosystem of curated partners, all the way to the explainability
03:56that Swami and Mike were talking about with features like our audit trail, citations,
04:03just really helping customers and end users understand exactly every step of what's happening to produce an output or automation
04:13of any kind.
04:14So, it really starts there. In terms of where we've deployed it, it's really been across the board from sales
04:21cloud, service cloud, marketing cloud, commerce cloud, Slack.
04:25I know there are a number of Slack organizations here.
04:28And I think one of my favorite stories was, you know, two years ago when our team started working with
04:35Gucci, one of our favorite European customers.
04:40And Gucci, like many retailers, wanted to ensure that their customer service team was providing excellent service for every client,
04:51especially given that luxury clientele.
04:54And to do it in a way that was really informed by data, but rooted in personal connection and deep
05:00relationships with a client.
05:02And so, over the course of six to 12 months, we really pioneered what has now become our Einstein AI,
05:11generative and predictive.
05:12We've had predictive in our product for almost a decade, but on the generative AI side, a way for us
05:18to help Gucci service representatives in the moment while they're engaging with their client,
05:24you have AI assistant to help them resolve questions and issues much faster.
05:30Now, the fascinating thing is that we thought, I think both Gucci and the Salesforce team, we thought that this
05:35would just reduce the length of the customer interaction.
05:39But actually what's happened is that instead the client service representatives have taken on a bigger role.
05:46And they've started, they've gone from providing only customer support to now also becoming brand storytellers and salespeople.
05:54And so, it really is a great story of augmenting and empowering employees to do the best work in their
06:01careers.
06:01And it's always inspiring for me when I talk to the Gucci client service representatives that are using Einstein.
06:08Okay. Swami, do you have an example, but not about Gucci?
06:13I can't be that fashionable, but okay.
06:17I'll say, I mean, first, there are two kinds of examples that pop to mind, but I'll be quick.
06:24One is we have a service called Amazon Bedrock, which is used by tens of thousands of customers, AWS.
06:33And the goal is to make the Gen.AI application development easy.
06:39But one of the hardest challenges is it's not just about guardrails layered on the model.
06:45When I am in a company building my own application, I wanted to actually stick to topics that are relevant
06:52to the application I'm building.
06:54I don't want it to talk about, like, competitor's product or actually, if you're an investment company, you don't want
07:00it to give, like, financial advice and without actually explicitly having the right disclaimers and so forth.
07:08So, Bedrock actually has a feature called Amazon Bedrock Guardrails.
07:14Guardrails is an easy way to deploy, but the company application-specific guardrails, and it improves the efficacy of these
07:24guardrails by up to 85% by turning this on.
07:28And we have several of our customers already deploying it for their application-specific guardrails so that they can say,
07:36hey, I'm a fintech company, I would like to actually deploy these.
07:40When I get questions like these, I should not be answering, or I should have these disclaimers.
07:45Or if I'm a consumer company, these are the PII information I should not be taking into place, and so
07:53forth.
07:54So, these are examples of how you build safe and trustworthy Gen.AI application.
07:59On the predictive side, we have a lot of more than 100,000 customers build machine learning models using SageMaker,
08:09and we collaborate with Salesforce on it, too.
08:12We have a product called SageMaker Clarify, which actually helps interpret these results on the machine learning side.
08:21And it is used by the likes of Thomson Reuters or NatWest to understand why certain predictions are the way
08:29they are.
08:30And it's super important, especially when you're trying to make sense of predictive results and understand why these models are
08:38generating these results.
08:40And these are just a couple of examples of how do you build trust into AI applications so that we
08:46can actually deploy them at scale.
08:49And for you, Michael?
08:50Yeah, just maybe briefly, you know, we produce foundation models on which really anything can be built, and so there's
08:57sort of two layers to this.
08:58One is, what is the kind of research and testing that we can be doing at the base layer to
09:03make sure that the guardrails are correct,
09:05that customers understand how to prompt the models in order to reduce the possibility of bias or considering factors and
09:14sort of decisions that they're not supposed to, and so on.
09:17And then the second piece is working really closely with customers and partners to develop and implement really good ways
09:22for them to test to make sure that the model is actually right for that application area,
09:27that it has the right set of capabilities, that they understand what the limitations are.
09:31And we're doing this across healthcare, legal services, chatbots, customer service, really across any particular use case where you might
09:39want some kind of augmented intelligence,
09:41intelligence, the models are being used for that.
09:43The question really is so important, though, just because, I mean, Swami and I were talking in the green room.
09:50Generative AI in particular, it's inherently non-deterministic.
09:53And so, being, and also, to a deep extent, many of us, no one knows exactly how transformers actually work.
10:02And so, to the extent we can, isolating the variables, really trying to understand before we deploy these systems into
10:09production, is absolutely paramount.
10:13And, Michael, you have, like, this thing called constitutional AI, it's a specific approach you develop, can you just tell
10:21us what that means exactly in practice?
10:23Yeah, well, let me first jump on something that Clara just said, because we're really excited about some research results
10:27we released yesterday in the field of mechanistic interpretability.
10:32So, often, because these models are non-deterministic, and it's hard to understand what's going on, people have wanted to
10:38talk about, like, explainable AI.
10:39Can the model generate some kind of plausible explanation for a particular output?
10:44Mechanistic interpretability is about understanding what's actually going on inside the model.
10:48I sort of call this, like, reading the mind of models sometimes.
10:52And yesterday, we released a paper finding some really exciting results on a full-scale production-sized model,
10:57where we're able to understand these patterns of the way that the neurons in the model activate.
11:03We call these features.
11:04And they're really tied to how the model understands different concepts.
11:08So, there's a feature for understanding the Golden Gate Bridge.
11:11There's a feature for software vulnerabilities.
11:13There's a feature for gender bias.
11:15And by adjusting the activations of these features, we can see that the model responds differently.
11:21It might write software with more bugs, or it might write software with fewer bugs.
11:25And so, we're really understanding now what's going on inside the model in a way that hopefully
11:30is going to allow us, in the long term, to steer behavior, and in the short term, find some safety
11:35risks.
11:35You know, we've seen features that are tied to generating misinformation, for example.
11:39So, a really exciting paper that just came out yesterday.
11:42On constitutional AI, you know, there's sort of two main stages in building an LLM.
11:47The first stage is the pre-training.
11:49This is the thing that costs, you know, tens or hundreds of millions of dollars, maybe billions in the future.
11:54And then the second is the fine-tuning stage, where you'd really teach the model to behave the way that
11:59you want it to.
12:00And traditionally, the way that companies have done this is using reinforcement learning with human feedback,
12:05where you hire a lot of crowd workers, and then they sort of give the model in the A-B
12:09test feedback on how to behave.
12:11What do you do when somebody asks you to help you do harm in the world?
12:15Or what do you do when you come across a situation of racism, for example?
12:19And this is effective, but the challenge with this technique is that you end up getting the implicit values of
12:25the crowd workers that you hire.
12:26And so we wanted to see, how could we actually be very explicit about the values that we want our
12:30models to uphold?
12:31And so we developed a technique that we call constitutional AI, where we replace a lot of the human feedback
12:36with another model that teaches the large pre-trained model how to follow a constitution.
12:43And the constitution has principles derived from the universal declaration of human rights, from other international statements of AI principles
12:51and ethics.
12:52And in this way, you can be very explicit about the principles of the model that should follow and what
12:57values you want.
12:58And then maybe just for one other quick thing, last year we also wanted to see, could we source public
13:03input?
13:04Could we collectively source input into this constitution?
13:07So we recruited a panel of roughly representative Americans and had them look at our constitution, debate it, suggest their
13:14own ideas.
13:15And two interesting things happened.
13:17One, there was a lot of overlap between what they thought should be in a constitution and what we found.
13:21And two, they surfaced some new ideas that we hadn't considered, such as making the technology more available or accessible
13:27to people with disabilities.
13:29And so we took some of that input from the public panel and we incorporated that into our new Claude
13:333 model that we released recently.
13:35And we found that it actually made the model better.
13:38So normally there's a Pareto trade-off between making models harmful and making them helpful.
13:43And we found that we actually got the models to be both more helpful and more harmless, which was a
13:48great finding.
13:50And you also have like AI principles, maybe how do you, yeah, how does AWS, you know, trying to ensure
14:00that AI systems are aligned to those principles?
14:03Yeah, so if you look at AWS, we build lots of AI systems to be used internally, but also so
14:14that our customers, the millions of customers are able to build using them.
14:19And we typically look at these in like eight dimensions.
14:23I'm not going to talk about all eight dimensions here.
14:26I will just highlight a few of them.
14:28One is around fadness.
14:30Fadness is such an important thing when it comes to deploying these systems because they are implicitly, you have to
14:37be careful about how these inferences are being made.
14:42And how do they affect in terms of the diversity of the data set and its impact among gender or
14:50race and various other aspects of it.
14:52The second one is around what I would call as like explainability.
15:00Typically, when you look at the early days of machine learning, which is like linear algorithms or gradient boosting trees,
15:09the models were extremely explainable.
15:13Whereas when you look at actually deep learning, I touched on things like SageMaker, Clarify, and Wardan.
15:20We were able to make enough progress on these models.
15:25But the third aspect, but with generative AI, the research is still evolving.
15:30Michael just touched on what Anthropic is doing.
15:33And we have similar such investments on understanding the activation.
15:38But the other dimension which we invest a lot is how do you, in a generative AI world,
15:43how do you ensure the content that is produced is actually robust and verifiable?
15:50What is actually a new original content anymore?
15:53So what are the mechanisms you're going to use in terms of watermarking or fingerprinting?
15:59And that is an example where I'll just use, we have an image generator model called Titan Image Generator.
16:06And there, all by default, all are images that are generated are watermarked.
16:14So that when you actually see an image, you can actually verify if this is AI generated or if it
16:20is real.
16:22That is like an example of how you essentially make all these systems to be easily adhered to certain responsible
16:32AI principles as well.
16:36Yeah, and from a Salesforce standpoint, you know, it really starts with the technology.
16:41We talked about that earlier with the Einstein Trust layer.
16:45Then we have our AI acceptable use policy.
16:48And so how can these AI capabilities be used?
16:53And so, for example, we never allow facial recognition technology to be used in our platform.
17:01And then at the policy layer, and this is something that, you know, our public policy, our AI ethics, and
17:09our legal teams have collaborated on and open sourced with a number of other organizations and government agencies, is our
17:17AI ethical principles.
17:19And we have five.
17:20So it's about accuracy.
17:23It's about honesty.
17:24So when the AI doesn't know, it doesn't make it up.
17:28It's honest about that fact.
17:30It's about empowerment of human employees and customers so that we're not replacing people.
17:36We're empowering people to do their best work, just like in the example that I shared before.
17:41It's about transparency.
17:42It's the explainability piece.
17:44And then it's about sustainability and making sure that we're using the right size model for the right size task
17:51and being conscious of our climate impact.
17:54So I wanted to talk now about governance, about regulations, of course.
17:59Sorry, that important topic.
18:02Michael, I saw that Entropic has been involved in a lot of talk.
18:07White House voluntary AI commitment, the UK safety summit, it was in November last year, G7 conversations, a lot of
18:16things.
18:16And, of course, in Europe, we have the AI Act, you must know, is almost adopted fully.
18:22How do you see regulation keeping up the exponential pace of development of AI, of this technology?
18:32Yeah, it's a great question.
18:34And I think Entropic's really been at the forefront of helping to inform and develop and implement a lot of
18:40these self-commitments, self-regulatory commitments that have been made at the White House, at G7, and elsewhere.
18:46And actually, yesterday, at the AI Safety Summit that's being held in Seoul, a new set of commitments was released
18:54where about 16 companies, including Entropic and AWS, have committed to developing something that looks very similar to Entropic's responsible
19:02scaling policy.
19:03So we were the first company to put something out like this last year, and now other companies have released
19:10theirs, and now we have 16 companies from around the world, including Europe and Asia and the Middle East, that
19:15are making similar commitments, which is really great.
19:18But the way that we're going to be able to regulate the technology effectively in a way that balances the
19:25benefits of the technology, the need to innovate, but also addressing and mitigating the risks, is by developing really good
19:31ways of measuring the capability and safety characteristics of it.
19:36Right now, we don't really have very good ways of evaluating the models. Benchmarks that are out there in public
19:41are not very good. It's very easy to game them, and it's also, you basically have to trust the companies
19:48when they say their models perform at a certain level on a benchmark that they're giving you the right information.
19:52And so what Entropic really wants to see is the development of an ecosystem of third-party model evaluations.
19:59Right now, for example, if you want to buy a car, you can go to a car dealership, and you
20:05can see how those cars perform across a variety of metrics.
20:08What's the fuel economy? How does it do in different kinds of crash tests?
20:12And in those areas, we have third parties that conduct the tests and evaluations, and so you can trust that
20:18if it says it's a three-star or a five-star crash test rating, that's actually a reliable indicator of
20:24how safe the vehicle is.
20:26There's nothing like this in AI right now, and we really need to get to the point where you can
20:30go to third parties, have them test and publish results of your models.
20:34And part of the reason why this is really important is because right now, a lot of the benchmarks are
20:39focused on capabilities.
20:41As a safety company, we want a lot of the benchmarks to also be focused on safety.
20:45This is important for customers and governments and the public to have trust in the technology.
20:50And so imagine a world where you can get your models benchmarked across a variety of capability and safety characteristics.
20:57These benchmarks are published, and so there's kind of a leaderboard of results.
21:01That's going to create a situation where there's real incentives to making models that are safer and more reliable because
21:07you're going to want to be at the top.
21:08Anthropics is going to want to be at the top of that leaderboard and safety, and so we see this
21:12as a way of creating a safety race to the top.
21:15And then finally, the last piece is, this is also a way of making the market more competitive.
21:20Right now, a lot of decisions are driven based on brand perception or who's sort of got the largest marketing
21:27budget or something along those lines.
21:29But in a world where everybody can compete based on quantitative metrics that are independent and reliable and objective,
21:37startups and smaller companies will be able to compete just as effectively with the largest names in the industry.
21:43And I think that's a good thing.
21:44It's certainly a good thing for the innovation ecosystem here in Europe, and we hope it's a good thing for
21:49safety more broadly.
21:51And for you, Swami, how do you address these challenges of responsible AI development across all the regulatory landscapes?
22:03First of all, given the potential of AI, AI has the potential to disrupt every industry in a big way.
22:12And we won't fully realize its potential unless you actually, you shouldn't just focus on technology alone.
22:21You've got to actually work with policymakers to make sure they have a view on how to actually deploy these
22:28systems in a safe and responsible manner for various systems.
22:32And this is where you've got to actually take a risk-oriented approach because there are certain use cases where
22:38they can be extremely high-risk systems,
22:41where at the end of the day, machine learning systems are like predictive systems.
22:46They are not making decisions, and in high-risk systems, you should not be using them to make decisions in
22:53an automated fashion.
22:55So this is where now we actually, going beyond technology, we work with policymakers around the world.
23:01And I personally sit in a personal capacity on the National AI Advisory Committee in the U.S. advising the
23:10president.
23:10And we work with the White House on their AI voluntary commitment, where we actually were one of the first
23:19few companies to really sign up for it.
23:22And we also, and out of that was an example of an effort on watermarking all the generative AI-based
23:30images, watermarked by default.
23:32And of course, yesterday, we also participated in the Korea AI Safety Summit, and before that, on the UK Summit
23:40as well.
23:41These are some of the examples of, if you go back and look at the early days of cloud computing
23:47or internet,
23:48in the early days of these kind of disruptive technologies, technologies actually go extremely fast.
23:56And you don't want them to thrive, but you also want to make sure regulators understand how to actually deploy
24:04them safely without curtailing innovation.
24:07That is why it is super important for all of us across the industry, academia, and policy landscape to collaborate
24:15so that we actually can keep up with innovation while being extremely safe and responsible.
24:23I'd say regulation is very important to protect consumers and to foster healthy competition and innovation.
24:31So there's a very important role to play.
24:33And similar to AWS and Anthropic, the Salesforce team, we've been very active partnering with, educating, and collaborating with governments
24:42from around the world, including here in France and in the EU.
24:47It's very early.
24:48You know, this is like 1998 for AI, and I think there's two important lessons to learn from that.
24:55First is that there's a lot of opportunity and risks that are yet to be seen.
25:00So this is something that we can't just have one conversation about how to regulate AI and then we're done.
25:06AI itself is constantly evolving, and so it's something that we have to be very agile across public and private
25:13sectors to continuously build together.
25:15I think the second lesson to learn is that, you know, there's also a risk of going too slow, and
25:22you know, in 1998, if you had halted rolling out the internet, then that economy, that company would have lost
25:28out.
25:28And so I think there's a lot of upside, and there's going to be, there are unknown unknowns that we
25:34have to learn together.
25:36The last thing I would say is, you know, regulation typically focuses on what not to do.
25:41And certainly that's important to outline, to take a risk-based approach on what not to do, to leverage and
25:48to update and modernize existing laws to make sure that they reflect the new possibilities and risks that come with
25:56AI.
25:56But equally important to what not to do is, we need to talk on the policymaking side in every jurisdiction
26:06around the world about how should we be updating our government?
26:11How should we be updating our education systems?
26:13If you think about what we're teaching our kids in school today, that was for a different era.
26:18It wasn't even for the internet era, and so how can we be preparing tomorrow's workforce and tomorrow's citizens to
26:25thrive?
26:26And I think that dialogue, I haven't heard as much of, and it's a fantastic opportunity for us to be
26:32able to engage in that dialogue.
26:34Okay. And very briefly, if you can give just one piece of advice for the companies here that want to
26:42adopt responsible AI.
26:45Michael?
26:45Yeah, well, I think the most important piece of advice is really choosing the right partners.
26:50The companies, Salesforce, AWS, and Anthropic, we've all talked about our responsible policies, how we work with customers to make
26:58sure that the use cases are correct for their deployment, that we're testing and evaluating the systems properly, and that
27:04we have the shared responsibility to protecting user and customer data.
27:08And so you really have to find the right partner that shares your value and shares your same commitments in
27:12order to adopt the technology appropriately.
27:16I would say, think of safe and responsible AI development in the end-to-end development of your application lifecycle.
27:25It is not just enough to think about securing in one aspect at one layer.
27:30It is, at the end of the day, it goes from all the way from the core model layer guardrails
27:36to how do you, what kind of data goes in, how do you evaluate, and what kind of application guarantees
27:43you're doing, and the risk level.
27:45So, you need a more end-to-end integrated approach, and if you do that, I think you set yourself
27:51up well in the future.
27:55Thank you, Clara.
27:55My advice to individuals and companies would be to adopt an agile growth mindset, because AI is changing every day.
28:05You know, every week there's a new research paper that comes out that changes everything.
28:08And so, expect that whatever we develop now will become obsolete in a few weeks or a few months, and
28:15just that practice, that muscle memory of continuously learning, continuously adapting, and evolving.
28:23It's hard to keep up, yeah, thank you very much for sharing all those experiences and examples, that was very
28:33great.
28:33Thank you, thank you all.
Commentaires