- il y a 2 jours
Shared Progress: How Can Open-Sourcing Speed the AI Race?
Catégorie
🤖
TechnologieTranscription
00:00Hello, everyone. Hello again, if I saw you already today.
00:05So, this session, we're going to be talking about open source.
00:08So, just to set us up, AI is, you know, the new infrastructure of our world.
00:14It's running in the background of companies, healthcare, education, all sorts already.
00:19But who gets to decide how it's made, how it's used, how it's changed?
00:23Are we happy to let just big tech companies decide how it is developed in the background?
00:31With me, I've got a stellar panel of experts with all from different perspectives on this interesting challenge.
00:39With me to my right is JB Kempf, CTO of Scaleway and the developer of VLAN Video Player.
00:48We have here also Stefano Maffali, the director of the Open Source Initiative.
00:53And over on the far side of the stage, hello, is Laurence Solli, is VP of Europe for Meta.
00:59So, all come at this from different parts of the spectrum on open source.
01:04But I'm going to start with you, Stefano.
01:06Obviously, the Open Source Initiative has been involved in sort of deciding what counts as open source for many, many
01:12years.
01:12So, tell us, why is it important?
01:13Why are we talking about open source and how would you define it?
01:17Thank you.
01:19So, the Open Source Initiative is the international charity that guarantees that software developers and users have unrestricted and freedom
01:29to innovate without having to engage or to ask for permissions to the rights holders.
01:34That is a very basic principles that has been around for multiple decades since software started to become a thing
01:43that you actually use in the late 80s and 90s.
01:47When software was starting to be produced and commercialized, if you want, to some extent, there were people who thought
01:55about the fact that they wanted to build on top of what others were doing.
02:00And they wanted to do it in a way that was unrestricted.
02:04And as an effect of that, we have the internet.
02:09It was this unrestricted freedom.
02:12It was permissionless and frictionless innovation that is embedded into the principles that our organization oversees and maintains is what
02:21has allowed for JB to write video land
02:26or to corral a community, to write a very popular client, to read, to watch movies, for example.
02:35Or what Mark Zuckerberg could do in his dorm rooms with developing the Facebook, the original Facebook, right?
02:44It was that kind of asset, right?
02:47Those principles are the basis, and that's why we're so extremely protective of those principles as open source initiative,
02:56because we want to make sure that no one, large corporations, small corporations, or even regulators,
03:03can start introducing nuances and interpretation of saying, well, you know, this open source thing, it's open source,
03:12but, you know, you need to ask for permissions, or you can only use it if you're not European.
03:17You can only use it if you're not a cloud provider, right?
03:20These may look on a very fast, rapid scan as open source, but they're not,
03:26because they are preventing you from having that unrestricted freedom.
03:31JB, why does open source matter? Why do you care?
03:35Well, I do care because I've been doing that for 20 years.
03:38But also, it's like the way of the future, right?
03:40Like, everything will be, like, software by definition is something where the aggregate sum of the next license is zero,
03:49right?
03:50So, software cost tends to go to zero, right?
03:53And everything that is infrastructure is not where there is a value added, right?
03:58So, like, if you look at meta, I'm quite sure that 95% of the source code compiled on the
04:05meta servers is open source.
04:06The value of meta, of Google, of all the large companies is on the 5% of the actual application.
04:13All the frameworks from the Linux kernel to the BMCs to the Python and frameworks and so on are getting
04:20open source.
04:20And AI is a new change.
04:22It will become the same, right?
04:25Everything that is common infrastructure will get open source.
04:28And this is a fact that you cannot avoid, right?
04:32I'm quite sure about that.
04:33And I've been telling that, like, quite a long time ago.
04:37And people say, no, no one is going to arrive at the stage of chat GPT and so on.
04:43And then arrived several open or pseudo-open models, right?
04:52From Lama and then Dipsick and so on.
04:54And you realize that all the open-weight models, the ones that are based on open source principles, are catching
04:59up, right?
05:00And when you see, well, the difference and the top of the board, there is so much innovation and there
05:04is so much at stake
05:05that open source is the way of doing AI and everything that is on AI, whether it's on training or
05:13inference, is based on open source framework, right?
05:15So the two are tight, right?
05:17And I think that the three of us agree on that major idea.
05:23The question then comes to the details.
05:26But there is something that is quite important for us Europeans, is that the only way that Europe will be
05:31able to not be left behind is to use open source,
05:35because we don't have the resources, but we know how to work all together as commons, and open source is
05:40key for that.
05:41All right.
05:42Thank you.
05:42Laurent, okay, how does Meta, what's Meta's approach to open source?
05:48Where do you stand?
05:49Yes, the company has been committed to open source for decades, in fact.
05:54Of course, we're going to speak about AI and the open source model, but I remember a story when 11
06:02years ago, a bit more,
06:03we opened our laboratory research in Paris, which was FAIR, Fundamental AI Research in Paris.
06:11The first commitment to Mark Zuckerberg and Jan Lekin was absolutely to make this laboratory open source.
06:19It was a deep commitment, because at that moment we believed that it was a massive argument to spread the
06:29research on AI, to accelerate the division of AI in further scientific community, the research community, the academic community, and
06:40to make all these things open and available.
06:43And if you take the story of the last 12 years, not only for Meta, speaking with humility, but we
06:51deeply believe in open source.
06:52And this is the same with AI, and as it was mentioned, our LLM Lama is open source, free and
07:01available for a lot of people.
07:03And I think this is one of the important commitments of the company, and this is something we deeply believe
07:11in, and as it was mentioned by my colleague from Scaleway, we also believe that this is a huge opportunity
07:17for Europe.
07:19That open source, we have seen, I think all of us in the last, let's say, five, six months, is
07:24suddenly developing its impact in all the continent and economy in the world, and we believe that's also for Europe.
07:31This is a huge asset to accelerate on AI and to be in the race, in the global race of
07:37AI.
07:38Can I jump in?
07:39Yes, I saw your smile across your face.
07:42Yes, because this is one of the conversations I've had with Yelne Kuhn, with a lot of the people at
07:49Meta, is that LLM Lama, it doesn't really get anywhere close that frictionless, permissionless innovation layer.
07:59It's available, it's made available to many people, like you said, many people, but you exclude it, for example, it's
08:04excluded from the usage, every European company or individual.
08:09So that's the kind of difference that we get into, like, eh, you know, not quite, but not enough, right?
08:17But at the same time, you know, Meta has contributed quite a lot of software, quite a lot of other
08:22things, like PyTorch, React, you know, those are pieces where we're not really picking on Meta, we're just trying to
08:29be protective of the principle of open source.
08:33Yes, we are partnering with open source initiative for years, and sometimes we have debate, and I won't say disagreement,
08:40but sometimes debate on the definition itself of open source.
08:44And it makes sense.
08:45The way this technology accelerates so fast, we have that debate on that.
08:50Nevertheless, and I like your point about also how open source can change the world and accelerate in Europe.
08:56One of the debates we have had in Europe in the last month was not exactly on open source.
09:00It used to be, it was, but it was more on the regulation we have in the European Union.
09:05And I think it is a key debate, not so far from the open source.
09:08If open source is in this country and this country and a huge opportunity to develop companies and to help
09:16companies to lead the world on AI, we need also the good regulation to do that.
09:21So one of the difficulties we have had in Europe to release our latest model, let's say the multimodal one
09:28in Europe, was not exactly linked to the way we see open source.
09:32It was more linked on the specific regulation and the rules that was in the EU.
09:37It was not only the case for Meta, it was the case for OpenAI, it's been the case for Google,
09:42etc.
09:42And it's a very interesting debate, not only from the tech company, but also currently from all the tech ecosystem
09:50in Europe.
09:50And I will say from all the European economy to face on the topic, in the same way, if we
09:56want Europeans to be in the race, they have also to be maybe suddenly less regulated than it is or
10:03it used to be.
10:04Sure, but even before the European regulation, there was a limit, for example, of $700 million to be able to
10:12use some of the models and so on.
10:14And like one of the key freedom of open source, one of the four freedoms is freedom to use, right?
10:21And the regulation changes, the software doesn't, right?
10:25So like what you can do is have a license that says you can use it as you want.
10:30However, be careful, European regulation doesn't allow you to do that in your own case, right?
10:37Because here you're pre-shorting it, right, the regulation, which I understand because you, as a company, you care about
10:45like regulatory dangers.
10:46But as a developer and from the open source standpoint, like give me the weights and if I violate my
10:54own country law, it's my fault, not you as a provider.
10:58Yes, and also when we see how LAMA has been downloaded in the last years, when we see the impact
11:05on the SMBs, for example, the startup ecosystem, the tech ecosystem, the feedback we have.
11:12is that the way we open LAMA to that kind of communities, the way it has been downloaded massively, more
11:20than $1 billion in less than one year, I think is also a proof of the commitment on open source.
11:27We can have the debate on the bigger platform and the license you're mentioning.
11:31I think one of the things which is mostly important, and I see that, and we see that every day
11:36in Europe, talking to all the clients, the startup ecosystem, the tech ecosystem, and the SMBs in Europe, the way
11:44they are using currently our LLM, the way they are downloading, the way they are fine-tuning, the way they
11:50take advantage for this technology is massive.
11:53And we all know that they can't do that themselves.
11:56Of course, we can have a debate on a few points, and I totally respect that point.
12:00Nevertheless, when you see the impact in the last just one year of this open source commitment, open source philosophy,
12:08the feedback we have from all the communities in Europe and elsewhere in the world, I think this is a
12:13strong commitment from my company.
12:15And I think all the Meta guys are quite proud of this commitment from Meta for years.
12:22Okay.
12:23Well, let's just take a couple of steps back, because I feel like we could do this bit all day.
12:29Maybe we will.
12:30But what's the business case for a company like Meta to decide to be open source versus open AI or
12:39Anthropic not?
12:40Where's the decision made, and where's the business case for doing so?
12:47You know, it's been a debate for years on that, about how we develop AI.
12:53The first thing, before speaking about the model itself, is the way we use our AI development internally.
13:00I want to take the time maybe to mention that, because of course this is an open source and it
13:04is free for all the people.
13:05The first execution or application of AI for us started, let's say, in 2016, before we spoke about Gen AI
13:14and before people knew that ChatGPT will come.
13:16The first impact is on moderation.
13:20The way we moderate the platform, the way we stop automatically, proactively, all the inappropriate content is due to AI.
13:31And it's very, very important, because sometimes we speak about AI as a risk.
13:36The first asset of AI for us is the protection of the platform on the moderation point, which is absolutely
13:42critical.
13:42The second point is that currently, AI is part of the business model of Meta itself.
13:50If you are an advertiser, and you are using our tools, a large part of the advertising tool from Meta
13:57right now is fueled by AI.
14:00Automatically, some media campaigns are created, some targeting communities are targeting, and the level of ROI impact for the company
14:09itself is massive.
14:11And we know that when you are using AI tools for advertising, when you are spending one euro, your ROI
14:19is up to four and more.
14:21So, to your question, the first business impact is for Meta itself.
14:26Because we need, this is a commitment, to give to all the clients in the world, to all the media
14:32agencies in the world, to all the SMB tools of you as users, a safe place.
14:39Well, you can find your friends, you can share with your friends, and if you are an advertiser, a company,
14:44you want to find your client, you need to have a safe place.
14:47What we call in the advertising industry, brand safety.
14:51This is due to AI.
14:53This is the highest standard of protection.
14:56And after, if you want to have the better advertising tool to send the good message to the good people
15:02at the right moment, AI is massively helping you.
15:06With the highest ROI, this is the business impact.
15:08After, for the rest, sorry I'm a bit long, but for the rest of the world, why open source makes
15:13sense?
15:15But if we spread the innovation all over the world, if it goes as fast as it goes, I think
15:22thanks to AI and not only AI open source and not only with Meta,
15:26but this is a chance also for all the community to take the advantage of the technology.
15:33If we look back to the last 30 years of the digital revolution, all the companies, all the communities, all
15:42the countries that have decided to commit themselves in this revolution,
15:46look, after 30 years, they all take the benefit of this technology revolution on their business itself.
15:53So this is our vision and philosophy.
15:56Thank you. JP, I feel like you're about to say something.
15:59Yeah, I'm a bit more cynical, sorry, but one of the reasons is that Meta is not a cloud provider.
16:07It's one of the large firm companies that is not a cloud provider.
16:10I know I'm one, right?
16:11And so they have a big case to have open models in order to avoid locking of some models on
16:22cloud providers.
16:24So it's great for Meta because they benefit from the ecosystem of open source and sharing,
16:31but it's also because their business model doesn't depend on it.
16:34And maybe I'm too cynical again, but if Meta had a cloud provider and wanted to do inference like Google
16:43for Gemini,
16:44Meta wouldn't have done that, right?
16:46So it's in their business sense.
16:48However, for me, Lama is interesting.
16:51It's interesting because it was, especially Lama 3, right?
16:55Because Lama 4 is a bit more controversial, was in terms of gain, one of the first shots saying,
17:03well, this is what open weight, right?
17:05It's not really open source.
17:08You can use it as you want.
17:10They give you the weight of the model, but you cannot train and you don't know how they train, right?
17:13So it's not really open source AI, right?
17:15It's open weight and you could call it stage one of open source AI, right?
17:19The second stage would be the data and what OSI has been doing.
17:23And the third stage is explain exactly how you train it and how you give the data.
17:28And very few people are doing the actual open sourcing, right?
17:31So there is a whole spectrum.
17:34And Meta with Lama 3 were the first one to short shoot everyone saying,
17:38well, this is a good model that is open weight and that is competitive with non-open models, right?
17:45So that was interesting.
17:45Of course, the next big shot was DeepSeq who managed to go even further with a fraction of the cost.
17:52And that's pretty good that Meta is like fighting against DeepSeq and others, right?
17:57Because in the end, the people who are end users are going to have a lot of models to choose.
18:03We have Mistral also who's doing like quite impressive small models, right?
18:08They are probably one of the leader of open weight small models.
18:12But again, in my opinion, as open source person, they are not enough, right?
18:18Can we clarify also the difference?
18:20Because this thing really rubs me the wrong way.
18:23And I got to spell it out.
18:25I got to say it.
18:26Mistral has a bunch of small models that are released with a license, with a contract,
18:33with a social contract towards the users and developers that allows them to do innovation
18:38without having to ask for permissions.
18:40Okay?
18:41The same goes with Gemma from Google.
18:45Same goes with the Fi models from Microsoft.
18:49Same goes with the Chinese developed Quen family of models or DeepSeq to some extent, even more.
18:58They release even more details.
18:59The one thing that is missing, the one model that is missing from this family that is quite different from
19:05these ones is Lama.
19:06I'm really sorry.
19:08It's because of that thing that if you are in Europe, you cannot use it.
19:12It's something that people tend to forget.
19:14It's not in the same class.
19:16I just said it.
19:17Sorry, just to dial one down again.
19:20What are the boxes you have to tick to be count as open source for you so anyone can access
19:26it?
19:26We did a bunch of work in the past couple of years to understand the very basic component.
19:33It's a little bit into the weeds, but I'll try to stay at a higher level.
19:37What you actually want to have to qualify as an open source AI, you need to have the weights, the
19:44parameters,
19:45available, made to you with basically responding to the principles of open source.
19:51You need to have access to also the complete code that generated the training data set and the training code,
19:58the actual training.
20:00And then you need to know exactly what data went into that machine.
20:04So it's a very high bar that is currently met by really a handful of organizations around the world.
20:12And that is Allen Institute for AI, LLM360, TII tend to do.
20:20Eleuther AI is another group.
20:21They tend to have these sort of models where everything is public, everything is accessible under permissive,
20:28permissionless innovation frameworks.
20:30And then there are a lot of other open weights, we want to call them, if that's the term, then
20:37we use it.
20:38You can see the parameters.
20:39You can see the parameters.
20:40You can use them.
20:41Right.
20:41You can use it without having to ask for permission.
20:43So for most of the end users, they don't care, right?
20:46Right.
20:46And that's important, right?
20:49Like, you have several states, right?
20:51There is really, like, closed source, right?
20:54There is open weight and what was mainly started by Meta.
20:58And then you have, like, the open source definition, which goes even further.
21:01And in my opinion, there is a few people who are trying to go even further.
21:05They give everything, training, data, model, inference, all the frameworks.
21:10And so far, almost no one is over there.
21:12You could call that Libre AI, right?
21:15Like, you have open, you have closed open weight, open source, and Libre AI, right?
21:20Mm-hmm.
21:20Okay.
21:21All right.
21:22Thank you.
21:22Thank you.
21:23Okay.
21:24Take it into, you know, so we don't get, you know, too mired in the definition side of things.
21:28But, you know, for example, Sam Altman will say, open AI can't make its models open source
21:35because it's just too dangerous.
21:37People will use it in bad ways.
21:39There will be, you know, terrorists.
21:41There will be, you know, all sorts of horrific stuff produced with it.
21:44It's full bullshit.
21:45Come on.
21:46Like, everyone has said the same things, right?
21:48Like, come on.
21:49Like, you could have said that for every innovation forever in the world.
21:52Like, that's just, like, he needs more money and raise no more money.
21:55So, he's going to claim that.
21:57But that's bullshit.
21:58And, like, the thing is, like, there is so many models that are good enough.
22:01If you're a terrorist, like, you have everything you need already.
22:04Like, blah, blah, blah.
22:07Well, for the other two, is there anything in, is any argument for keeping things slightly
22:14more closed?
22:15Do any of those arguments, like, ring true with you at all?
22:18Do you give some weight to them a little bit?
22:25So, like any technology has dual use.
22:30It could be used and abused.
22:32And there are risks.
22:33Like, there is a paper led by, well, quite a few authors.
22:39I'm, you know, one of the little ones at the bottom.
22:42But in the paper, we did look at all the risks that are possible.
22:47And there is really evidence for the risk of generating offensive images and revenge porn
22:55or fake porn.
22:57Those are real risks.
22:58Other risks, like creating bio-weapons or getting the instructions on how to build a bomb
23:03and all of that stuff.
23:04It's something that should be really taken with more than a grain of salt.
23:10Because there is no evidence right now.
23:12And it could be covered by other regulation.
23:15Like, knowing how to build a bomb is in any proper library.
23:20Any university course will teach you the basics, right?
23:24But then you need labs.
23:25You need access to components.
23:26And all of that is highly regulated.
23:28So the risks are less visible.
23:32What we need to be paying attention, though, I mean, one of the principles of open source,
23:36as a corollary or side effect of that freedom to innovate, is that you can really see how
23:40it's been built.
23:41And so that carries, almost automatically, a level of transparency.
23:45And with it, carries an increased amount of trust that you can put into that machine
23:51that is transparently described and available to more than one expert, you know, that can judge it.
23:57It works with software.
23:59Like, software security has been for many years now.
24:03It's taken for granted that an open source piece of software developed by a group of researchers
24:09and accessible to researchers is generally safer than an opaque or proprietary
24:16and never seen before piece of code, right?
24:19And so, hopefully, we're going to go into that direction with AI and machine learning.
24:26I agree with Stefano, and we have currently also a lot of debate in the community with other
24:32foundations about the perfect definition, and we take the feedback, we respect the opinion.
24:37On your question on open air, I prefer to let my colleague answer to you.
24:40So I won't go on that field.
24:45So I'm going to give you one reason, right, to stay close, is that it brings revenue, right?
24:54And so if you want to continue, right?
24:56So I'm going to say exactly the opposite of what I just said, right?
24:59But some companies, they want to have an edge, and there is a lot of business to do that.
25:04Because if you go fast enough, and if you're quite fast, then it's a business argument.
25:10It's not a moral argument, but it's a good argument.
25:13Because if you want to have the best researchers, they cost a lot of money.
25:17If you want to have the largest training clusters, which is something that you need, right?
25:22You need to have a business case to sustain that, right?
25:26And people like Meta can because they use it internally, and that makes them a lot of money on ads,
25:33right?
25:33But there is an argument of staying close because you want to make more money.
25:38And that seems like a pretty good argument.
25:40Of course, it's completely wrong.
25:42If you look at OpenAI, they are losing so much money that this argument is empty, in my opinion.
25:49And the business, as you just said, the business argument is an argument.
25:54I mean, you can go on stage and defend the way you see closed-source model for your own business.
25:59I mean, we're also a company, and we need to do good business to invest massively on that technology.
26:05And as you know, Meta is investing massively each year.
26:09And Scaleway also is driving a business, and it's super fair to see Scaleway making the best business in Europe
26:16to develop a great company.
26:18And I think it's why the business argument is, of course, an argument.
26:22I explained to you how we are currently using AI for us, so we have a different, of course, opinion
26:27and strategy on that.
26:29Thank you.
26:31All right.
26:31We've only got that much long left to go.
26:34I just want to try and sort of head us towards a conclusion of some sort.
26:37All right.
26:37Even if we go via sort of like a windy, meandering path.
26:41Where do you see the sort of landscape between open versus closed going over the next sort of year, 18
26:47months?
26:48You know, if you think of looking at like OpenAI and Anthropic are completely closed, and it seemed like they
26:53will always be.
26:53So there are then companies like Meta, and then Mistral.
26:57There seems to be also like a divide, like a national divide between Europe and the US.
27:03Where do you see this going over the next couple of years?
27:06Is it going to change?
27:07Are we going to see you drift one way or the other?
27:09What are your predictions?
27:11We'll come back next year and check to see if you were right.
27:14What?
27:15You want me to start?
27:15Yeah.
27:16Go on.
27:16Oh, gosh.
27:17Are you looking at me like the Oracle?
27:18Okay.
27:19I do have...
27:20You've got open source initiative in your title, so, you know.
27:23So I do have a crazy view.
27:27Like, I mentioned the three blocks of an open source AI, which are, it's the code that builds the training
27:34set, the training code, the model weights, and the data.
27:39And I do think that we're going to be seeing clusters of collaboration happening in all these three.
27:45Right?
27:46On the model weights, collaboration is already happening.
27:49Like, the ecosystem around Lama is massive.
27:52The ecosystem around DeepSeq and Quen are massive by themselves.
27:57So that is already happening.
28:00There is also going to be a lot more collaboration happening on the training.
28:04Because when creating a training, the training code is in any of these laboratories that are releasing these things, that
28:13all of them, the ones that I talk to, they all tell me the same thing.
28:16It's messy.
28:17Like, thanks to PyTorch, released by and stewarded by Meta, there is a common framework, or TensorFlow is another one.
28:26You know, there are quite frameworks, but the actual code that actually runs the training is messy.
28:31And it's a one-off in each one of these labs.
28:35I think this looks like a lot of the crazy early days when everyone was reinventing the wheels fast, rapidly,
28:41right?
28:41Because they needed to take the edge.
28:43The third piece, so I think that there's going to be more standardization, more collaboration, more organization of that framework,
28:51so that less duplication of effort happens.
28:54And then the third part, it's the most complicated and the one where I think that we need to move
28:59fast.
29:00And we need to move fast on two levels, on software, like on the technology front, but also on the
29:06regulation front.
29:07Data is where we have a completely new use for something that until two years ago, three years ago, we
29:15never thought about it.
29:16I'll give you an example.
29:17We were happily posting our images on social media.
29:23We were posting our blogs, maintaining our blogs on the web, and we had an implicit agreement with the corporations.
29:30Like, I give you my blog to Google to index.
29:34In exchange, I'm going to be discoverable.
29:36Like, I'm putting my pictures on Facebook, on Instagram, because I want to run, drive business towards my little store,
29:43or I want my friends to see where I am, what I'm doing.
29:46Like, but in exchange, we get some value.
29:49Now, all the data that we have been uploading and giving to the, unfortunately, they're all massive corporations in the
29:56most cases.
29:56They're taking and slurping all of it, and they're using it to create these tools, this software that is many
30:04times, oftentimes, proprietary.
30:06And the Googles, the Amazons of the world, they have machines that they keep ingurgitating our data to create these
30:15systems.
30:15And they're not necessarily, we don't have a way to force them to give us back the value that we
30:21have given them.
30:22So that is a social contract, in my mind, that has been broken.
30:26And I don't know exactly where the fix is going to be coming from, but it's both a technology fix,
30:31because we need to have standards to track provenance of that data.
30:34We need to know who the picture, who took the picture, where, who owns the rights for that picture.
30:39We don't have a way to do that right now.
30:41And we need to have the regulation that will be telling clearly to the corporations right now, they don't know,
30:49right, they don't know what they can or cannot do.
30:51They're being sued left and right. The moment they get transparent about, I built this AI using this data set,
30:59you know, there goes someone who holds the rights and sues them.
31:04Right? So there is a disadvantage right now for those who are completely transparent.
31:10And we need to fix that. And I don't know where the fix is going to be coming from. That's
31:13my prediction.
31:21I think the open source models are going to crush everything else, right? So besides open AI and anthropic, no
31:28one will do close model for foundation models.
31:31It's gone. I expect meta to be, to come closer to the true open source definition. I'm quite sure. I
31:39think their position is not really sustainable.
31:41To be honest, for the general public, this is a very small difference between us, right? It's a debate that
31:46is very small.
31:47But I think that meta, at least for Yama 5, will come closer to us.
31:53I expect, not in very close, but very soon, a new architecture on LLMs coming from the open source community,
32:00whether it's coming from Yann LeCun or DeepSeq.
32:02And this is going to change a lot. There's going to be like a big, because now we are at
32:08the point where we're optimizing LLMs, making them smaller and making them slower.
32:12I expect a big change. I don't know if it's in, I don't think six months, but in 18 months,
32:17something big is coming.
32:20And I believe that for once it's going to be the open source community or the open way community or
32:25us, right? Call it like that.
32:27That is going to change and not open AI for once. And I expect a lot of companies using open
32:36source LLM to be developed around the ecosystem.
32:39We were talking about Yama indexes or a lot of chatbots and so on. So I expect open source to
32:46take more on the product side, right?
32:48Like here we, and we don't have many open source companies on the product side, right?
32:54But it's, everyone is using open LLMs and open way models, but I expect a big change on that. So
33:01those are my three predictions.
33:03I'm excited to see what is the new architecture. Like, I feel like just a feeling you have in your...
33:08It's a feeling, it's a lot of papers. You should read what Yann LeCun is saying, right? Because probably one
33:14of the most clevers.
33:15But you should also see the change that we are done on the architecture around DeepSeq and everything around mixture
33:21of experts.
33:22I expect something like, which is not just going to be improvement in LLM, but like a small gap that
33:27is going to be quite interesting in the non-classical really language focused models.
33:34Oh, terrific. Thank you.
33:35Yes, I quite agree on what just has been said. First, we believe that open source has a great and
33:43massive positive future in the coming months.
33:46And I think the last six months has proven that. The second point, we deeply believe that this is an
33:52asset, a chance on innovation and for the economy itself.
33:57And we believe that we will see, we are seeing, but we will see more and more in the coming
34:01months, this positive spread and execution of all the open source model.
34:07I deeply think that we have also entered for the last month into that, what we call applicative AI moment.
34:14And I'm sure that next year, on that stage, we are more going to speak about the product of AI
34:19coming from the open source model, the development in strategic sector all over the world, I hope in Europe, et
34:27cetera.
34:27And that will be, I think, the point. So for us and for Meta, as we are open source, we
34:34are also, as you heard, open to feedback.
34:36So I'm sure that we will find the perfect definition of open source and open weight and maybe open science
34:42AI.
34:42There's a lot of definition going on. But I think what gather all together is that we are also committed
34:47to this vision of being open and accelerating the innovation.
34:50And we believe deeply that open source has proven in the last, let's say, six months and much more, that
34:55it will be the transformation force of the economy and the world itself.
35:00We can all agree on that, I think.
35:03I think so.
35:03Yeah. Good. Brilliant. Well, thank you so much for your time, you three.
35:07Thanks for having us.
35:07Really enjoyed the chat.
35:08Thank you.
35:08Thank you.
35:09Thank you.
Commentaires