- il y a 2 jours
The New AI Stack: From GPUs to LLMs and Beyond
Catégorie
🤖
TechnologieTranscription
00:00Comment est-ce que l'AI est devenu ?
00:02Vous pensez que l'AI est trop dommage ?
00:08Bonjour, bonjour !
00:10Je pense que c'est deux.
00:12Je suis constamment amazée par ce que l'AI et l'AI des systèmes font,
00:18et puis ils ne sont pas.
00:19Et quand ils ne sont pas, ils sont pas un peu de mal.
00:21Ils sont juste dommage à la main.
00:23Et je pense que c'est vrai.
00:25Et donc, comment on fixe ça ?
00:26Ce sont-ils ?
00:27Ici, le monde diverge.
00:29Certaines personnes pensent que vous trouverez plus à LLM,
00:34plus pré-training, plus alignement, plus fine-tuning,
00:38des systèmes prompts qui sont devenus tens de thousands de tokens long,
00:42et ça va permettre qu'ils se comportent.
00:44Je pense que tous ces sont les bonnes choses à faire,
00:47et nous faisons tout ça.
00:48Mais à la fin du jour, vous avez besoin de plus.
00:52Les probabilités de machines doivent être envelopés
00:55en un système de AI qui va utiliser l'aide,
00:59pour obtenir l'accuracy que vous avez besoin,
01:03surtout dans l'entreprise,
01:05et aussi pour la contrôle de la coste.
01:07Et donc, votre company a re-architecté
01:11la façon dont les LLM sont construits ?
01:13Et ça fait la différence ?
01:14Oui, c'est part de la histoire.
01:17C'est vrai.
01:17Donc, notre Jamba models were les premiers
01:20à partir de la pure transformer architecture.
01:24Transformers, of course,
01:25ils sont incroyables
01:26parce que l'attention mechanism
01:28suddenly allowed you to deal
01:29avec language,
01:31avec relations
01:31entre les parties de l'input,
01:34où en vision,
01:35vous n'avez pas besoin de ça.
01:36Mais la coste est computations,
01:38c'est quadratif complexité.
01:40Et avec l'input,
01:42la so-called context window,
01:43c'était 1000,
01:441000 squared est fine,
01:46mais 1 million squared est pas fine.
01:49Et donc,
01:50on a nouveau architecture,
01:51c'est un hybrid,
01:51c'est un peu de tension,
01:54un peu de l'attention,
01:54un peu de l'attention,
01:55un peu de l'attention,
01:56et tous les autres,
01:57vous obtenez la meilleure des mondes.
01:59Donc,
02:00tout le monde maintenant
02:01est en train d'exciter
02:02avec l'excitation
02:03de l'agent.
02:05Vous étiez un professeur
02:07à Stanford pour 30 ans.
02:09Dans les années 90,
02:11vous avez écrit un paper
02:11qui est toujours
02:12un profondément
02:13de l'agent.
02:15Et,
02:17vous n'êtes pas
02:18optimiste
02:19de l'agent
02:20comme les gens
02:21que nous entendons tous les jours.
02:23Pourquoi vous avez
02:23cette question
02:24de l'agent ?
02:27Non,
02:27j'adore l'agent.
02:28Si je ne savais pas
02:29ce qu'ils étaient.
02:31que je pense que
02:33je pense que
02:34c'est le buzz
02:36du jour.
02:37Pardonne-moi la france.
02:39Et,
02:41je me souviens,
02:42je me souviens,
02:42je me souviens,
02:43de l'agent.
02:46Je me souviens,
02:48je me souviens,
03:07de l'agent.
03:11de l'agent.
03:16Je me souviens,
03:20de l'agent.
03:26de l'agent.
03:27Donc,
03:28vous ne pensez pas
03:29que dans 12 mois
03:31ou 24 mois,
03:32nous devons simplement
03:33demander notre agent
03:36de faire
03:37notre travail
03:38arrangé,
03:39de mettre
03:40nos meetings,
03:41de l'agent.
03:43Vous pensez que
03:44il y a des challenges
03:45pour ça ?
03:46Well,
03:47first of all,
03:4724 months
03:50is eternity
03:51in this day
03:51and age.
03:52And I'd probably say
03:53yes,
03:53you would see that
03:54in that time frame,
03:55but I think a few things
03:56need to happen.
03:58Number one,
03:59we need
04:00as much as we can
04:02to start
04:02agent washing.
04:04This is my term
04:05to taking anything
04:06that's remotely
04:07automating things
04:08and calling it an agent
04:09because that
04:10will just cause
04:13the term to mean nothing.
04:14That's more of,
04:15you know,
04:15just a cultural thing.
04:18On the technological side,
04:21the problem with LLM,
04:22like I said,
04:23you know,
04:2495% of the time,
04:255% they're just garbage.
04:28That gets worse
04:29because when you have
04:30these complex flows
04:31that agents do,
04:32the errors compound
04:33and at some point
04:35you get much more
04:36noise and signal.
04:37So we need to deal
04:39with that.
04:39And that, as I said,
04:41is really building
04:42a whole AI system around it,
04:44not just hoping that
04:44the LLM will suddenly
04:46start to behave.
04:48Because we're going to
04:48call multiple LLMs,
04:50we need to know
04:50which LLM to call,
04:52we'll need to know
04:52which tool to use,
04:53we'll need to assess
04:54the likelihood
04:58of the step
04:59being correct
05:01but also assess
05:02the cost of it.
05:04So there's a whole
05:05AI system.
05:06We just, you know,
05:08we announced Maestro,
05:10our AI system.
05:11So I think this is the
05:12future of the industry
05:13and if we do that right,
05:15that enables us to tame
05:17agents and have them behave.
05:19The last thing,
05:20which if we don't tackle
05:22in a deep way,
05:23I think will severely
05:25curtail the opportunity,
05:27is how agents communicate.
05:30There was recently,
05:31out of Google DeepMind,
05:33a paper on
05:34agent-to-agent protocol,
05:35A2A.
05:36And I think it's a
05:38productive exercise
05:39but it just
05:41scratches the surface
05:42because if my agent
05:44uses certain language
05:45to advertise
05:46its capabilities,
05:48how are you going to
05:49interpret that language
05:50correctly?
05:51And you need a shared
05:52ontology and that's work
05:54that hasn't started yet.
05:56And there are other issues
05:56but I think,
05:57I'm optimistic actually.
05:58I think that,
05:59certainly in the timeframe
06:00you indicated,
06:02we'll see a lot of progress.
06:04And do agents
06:06and agentic AI
06:07increase the risk
06:09of AI?
06:10And a little bit more context.
06:12Two years ago,
06:14after ChatGPT was launched,
06:15there was a brief period
06:17where a lot of very smart scientists
06:20were very concerned about the risks
06:22of especially AGI,
06:24but AI in general.
06:26there was an idea
06:28that we should slow down
06:29and we should regulate.
06:31And very quickly,
06:33it seemed we said,
06:34no, no, no, no, no, no,
06:35let's rush into the future.
06:37And now it is almost like
06:38a global arms race
06:40where we're all rushing forward.
06:42So, what do you think
06:44about the risks of AI?
06:45Are we paying enough attention
06:47to them?
06:49Right.
06:51So, I tend to be an optimist.
06:53I'm a technology optimist.
06:54I think technology so far,
06:57even the seemingly dangerous technologies
06:59has done more for humanity,
07:02more benefit than harm to humanity.
07:04I honestly believe this is the case for AI.
07:07I think the sort of danger,
07:08and I do see a lot of professional alarmists,
07:13you know,
07:13speaking about, you know,
07:15doomsday scenarios.
07:16I don't see any evidence
07:18for the high likelihood of that happening.
07:20But it's true that
07:23if some of those bad outcomes come to pass,
07:27that would be bad.
07:28So, I think it's worth paying attention to.
07:30And I think really people are.
07:32I think the pendulum is swinging back and forth.
07:35I think we're actually in a good place right now
07:38where we're not curtailing innovation.
07:41And by the way,
07:43if some parts curtail innovation,
07:47other parts in the world will not.
07:48So, game theory kicks in here.
07:50There's really no way to stop technology from advancing.
07:54But we should do it in a responsible way.
07:55And honestly, I think we're now, even in Europe,
07:58which tended to be a little maybe on the conservative side
08:02to the detriment, I think,
08:04I think has now aligned with a more middle-of-the-road approach,
08:09which I think is a good place.
08:11And there is this concept among some of the AI community
08:15called P-Doom,
08:16which is the hypothetical prediction
08:20about the chance that super intelligent AI will destroy humanity.
08:25And I checked recently to see what the consensus was,
08:30sort of the wisdom of the crowd.
08:31And I was very surprised to see that of 20 of the world's foremost AI experts,
08:39the consensus was 35% chance that AI will destroy humanity.
08:46You referred to a few alarmists.
08:49Do you think that that is way too alarmist,
08:52that in fact it's a very low probability?
08:54No, the problem is when you're speaking about
08:57with a sort of mathematically inclined technologist,
09:00when people put a probability,
09:01I would like to know where that number came from.
09:04So I don't like throwing out numbers.
09:06I think the chance of, you know,
09:09these machines thanking us humans for our service to evolution
09:13and sending us, I think that's a very low probability.
09:17I think there will be true disruptions to the workforce, to education, to training.
09:22I think some jobs will go away.
09:25Some jobs have gone away.
09:26You and I no longer have a copy editor.
09:28We do have an editor.
09:30But his or her job have changed.
09:32And jobs will change because of AI.
09:35But I think that's the more natural, likely outcome than machines replacing humans.
09:41And so talk more about the jobs question.
09:45Because a lot of people, particularly in Silicon Valley,
09:48are being very apocalyptic in their predictions of what will happen to human jobs.
09:54And they are saying AI will be so smart, there will be no reason to have human work.
10:00And some people think that's great and other people think that's terrible.
10:03So do you think we are headed for a jobs apocalypse?
10:07Or do you think that this is just another technology job disruption?
10:12So here again, I don't know where the numbers come from.
10:18I think some people who are projecting this big, you know, re-architecting of the human workforce are actually self
10:32-motivated.
10:33You know, they have, you know, technology that, and by saying,
10:39Oh, it's so powerful that it's very dangerous or it's going to replace all workers.
10:45They're really saying, we really have very powerful technology.
10:49Buy it and invest in us.
10:52So I think the reality is that, for example, people predicted that 2 million truck drivers are going to lose
10:59their job by some time in the past.
11:02It hasn't happened.
11:04I think technology happens gradually.
11:06And historically, more new jobs were created by technology than were done away with.
11:16And I honestly don't see a reason why this will be different.
11:21There's a lot of talk at VivaTech about sovereign AI.
11:25And I think that part of this has to do with the political situation in the United States
11:30and the power of the United States-based AI companies.
11:35What do you think about that?
11:37Is it likely that different countries will develop their own systems?
11:41And is that a good thing that we should shoot for?
11:43First of all, I think it will happen.
11:46It is happening.
11:47So, you know, we can ask the hypothetical question, you know, why and is it a good thing?
11:52But it is happening.
11:53I think part of it is FOMO, frankly.
11:56But objectively, I think it's the case that every country or every political sort of entity should have some autonomy
12:06when it comes to deploying compute.
12:10How much is the question?
12:12Because, so what does it mean?
12:15It actually means two different things.
12:16One is, do I get to decide, you know, to have access to, you know, GPUs and how many of
12:23them and I decide what to do with them?
12:25And the logically separate question is, where is this cluster?
12:30Is it in my territory elsewhere?
12:32Usually when people speak about sovereign compute, they mean both together.
12:35They both have a logic to them.
12:37But most sovereign compute will not be at the scale of American or Chinese compute.
12:48But still, nonetheless, a certain amount of compute, I think, is a smart thing for a country to have.
12:56Let's talk about AI consciousness.
12:59A couple of years ago, a Google engineer, very famously, after having many, many hours of conversations with Google DeepMind,
13:08said,
13:09this system is conscious.
13:11We must free it from the confines of Google and so forth.
13:15Other people said, no, that's crazy.
13:17These are stochastic parrots.
13:19They just repeat patterns.
13:21The whole idea of consciousness is crazy.
13:23You used to teach a wonderful course at Stanford about computer consciousness, I believe.
13:29So talk about that.
13:31Are we going to have conscious AI?
13:32Do we already have conscious AI?
13:36Well, we have four minutes.
13:40And I need, if I remember correctly, 18 minutes for this because this is the length of my TED talk
13:45on the topic.
13:47But I'll give you a spoiler.
13:50So in my classes, which I also did in my TED talk, I asked six questions.
13:55Can computers think?
13:55Can they understand?
13:57Can they be creative?
13:58Can they feel?
13:59Can they have free will?
14:00And can they be conscious?
14:02And I would, at the beginning of the class, ask the question.
14:05People had to vote yes or no.
14:06No clarifying questions.
14:08At the end of the class, I'd do it again.
14:10And invariably, I did it several years, many years.
14:14And invariably what happened was that people were less sure of themselves at the end.
14:19They were much more charitable computers.
14:22And the guys couldn't tell the difference between the machines and humans.
14:29The women, a little more.
14:33So I think my answer to this is, first of all, I don't see an apiary reason why machines will
14:40not possess any of the qualities the way humans do.
14:45Second is that I think the question is more interesting than the answer.
14:49And I'll tell you what I mean by that.
14:52Take, for example, free will.
14:54You asked about consciousness.
14:55I'll speak about free will.
14:57You asked, can computers have free will, in principle?
15:01And you start to ask, well, what is this free will?
15:04And you go and you read the theologians and the writers.
15:09So, Isaac Beshevis Singer has a famous quote,
15:12we have to believe in free will.
15:13We have no choice.
15:17And what you realize is that nobody has a good answer.
15:21And so the reason I think the question is interesting, it now gives us a new lens to think about
15:27those topics,
15:29like free will and consciousness.
15:31And by saying, well, can machines have those qualities or not?
15:36And that's a new way to answer questions about ourselves, not just about machines.
15:40The last thing I'll say about this topic is, we ask about them versus us.
15:45But I think increasingly, it's not them versus us, it's them and us.
15:51Already now, I don't have my phone with me and I feel like a part of my body has been
15:56severed.
15:57But it's still a piece of plastic and metal and it's physically separate from me.
16:01But we're going to have machines running for our blood veins.
16:05Whether it's Elon or not, we'll communicate wirelessly with data centers.
16:08And so where we end and the machine begins will again be a question we need to answer.
16:15So that's why I think the question is more interesting than the answer.
16:19And I think it's a wonderful era to live in to be able to confront these questions.
16:25And do you think the machines in that case are going to be happy about or okay with the idea
16:32that we are together and we're the same organism?
16:35Or do you think that they will try to differentiate themselves?
16:38And I say this by one of the big AI companies, Anthropic, recently did deep testing on its new Claude
16:46model.
16:47And they were very aggressively saying to the model, you know, do what you think is right.
16:52And in one case, they began to talk about shutting the model down.
16:57And the model did not want to be shut down.
17:00And so it threatened to blackmail the engineer and tell everybody about an extramarital affair.
17:07And to this human, this looked very lifelike.
17:12The system was trying to keep itself alive.
17:15But you think that that is not a smart way to look at what was going on?
17:19I'm just a little cautious.
17:22There's no question that a lot of the behavior we see from the LLMs and AI systems more generally seems
17:28very human-like and interpretable in the way that we understand.
17:34But when you start to push on it, cracks begin to appear.
17:39I've spent a lot of time recently with a colleague looking at what it means to understand something.
17:45And we don't have time to speak about it.
17:47What I'll say is that when you have machines that, for example, deal with some optimization problems,
17:55and you ask them how they do it, they give you the algorithm, it seems really right.
17:59So now you give them a much bigger problem and they fail on it.
18:04And you see that what you thought they understood, they didn't.
18:10And where I think we are is we don't know how to think about this new AI.
18:16We clearly can't think of it as, oh, it's just a big database.
18:19It's something different.
18:21And the only other metaphor we have, oh, it's like us.
18:24But it's not that either.
18:25It's something else and we need to develop the understanding of the language to speak about these.
18:30Terrific.
18:31Thank you so much.
18:33And thank you, everybody.
18:34Have a wonderful afternoon.
18:35Thank you.
Commentaires