- 1 day ago
Category
🗞
NewsTranscript
00:00Really appreciate having both of you here, both obviously pioneers in your field.
00:05Jan, JP, you've both talked a lot over the last few years about sort of what our future really
00:12looks like when it comes to biology, when it comes to technology, AI, and LLMs. And I want to start
00:18specifically with you, Jan, some comments that you've made in the past and dovetail with some
00:23comments that we heard last night from Jensen Wong, who of course has been at the forefront
00:27of this build-out of this sort of LLM-led data center revolution. And he started to talk about
00:33this idea of an evolution beyond the traditional LLMs that we now know of to more physical AI,
00:40and the idea that how we need to start to think about the technology behind that has to evolve
00:46with it. I want to know if you can maybe address that, because this has sort of been the crux
00:51of
00:51what you've been doing. You've put a bill, raised more than a billion dollars, of course, for AMI
00:55labs in order to follow that. Are we now at that point where that transition is going to take
01:00place?
01:01It's going to take place over the next few months or perhaps years. But you very clearly
01:08see a trend, certainly in research, because the trends in research precede the trends in
01:14industry. Industry kind of trails behind a little bit. In research, you see a lot of people starting
01:20to get interested in what we call world models and physical AI and sort of dealing with real data
01:26as opposed to language or sequences of discrete symbols. I've been working on this for a very long
01:32time, you know, the better part of the last 15 years, essentially, and making fast progress over
01:38the last five, which sort of prompted the creation of Amilabs, the company I started recently. So you
01:47see that trend. I think 2026 is going to be the year of the world model. And you're going to
01:53see
01:53this type of model emerging more and more.
01:55Are you starting to see more investment in that space, Jan? Because we know that so much of the cap
02:01expense, certainly in the U.S., has gone more towards the traditional LLM data center model. I know
02:06things are maybe a little bit different in Europe and in Asia, but are you seeing more interest with
02:11regard to the money investors are willing to put towards your vision of AI?
02:16Oh, yeah, absolutely. And the reason is simple to understand. There is a lot of expectations that
02:23robotics is going to be a revolution, essentially, in a big market, eventually. But the secret of the
02:33industry is that none of the companies at the moment that are building robots, like human-arried robots,
02:39none of those companies has any idea how to make those robots smart enough to be useful. You know, domestic
02:48robots or, you know, robots in the industry that will have some degree of generality and adaptability. And so there's
02:57a
02:57big push for AI to make progress towards being able to deal with physical tasks, but first of all,
03:06understanding the physical world. And so that's what is prompting this investment. There's nothing wrong
03:13with investing in LLM. You know, LLMs are useful. They're just addressing a completely different set of
03:20applications, really.
03:21Well, when we talk about a different completely set of applications, I am curious about the work that you've
03:26been doing, JP, in the world of biology and the idea of what type of, forgive me if I get
03:32the phrase
03:32wrong, but what type of language models do we need where we can have better drug discovery and better
03:38research when it comes to developing and just simply finding out what's out there when it comes to
03:43biological advancements?
03:46Yes, absolutely. I think biology is exactly one of the fields that is going to be disrupted by AI.
03:52But you know, LLMs won't cure cancer. Cancer doesn't speak English, just like when you think of robots
04:00having to interact with the 3D world. When you're looking for a drug on cancer, the world is not the
04:06room,
04:07it's the cells, it's the molecules, it's the tissues. So what we need here is an AI that understands not
04:13English, but understands how biology works. And that's exactly what we're building at Bioptimus. As you can
04:20imagine, you know, the architectures, the way to train the model is pretty different from LLMs. But the high-level
04:26IZ is the same. If you have enough data and enough computing power, then you can train systems that, from
04:33the data, learn how the world of biology works. And that's what we need to move from trial and errors
04:40in medicine, discovery, to more rational design.
04:43Well, but as we're sort of trying to build the technology that kind of learns from physicality, biology, etc., JP,
04:50I am curious about sort of what actually is required, what more is required, whether it comes to the sensors,
04:56when it comes to the data, the compute, and really just the overall architecture of what you are trying to
05:02build.
05:03So there are many pieces in the puzzle. I think something quite specific in biology and in the models we
05:09build is a notion of multimodality and multi-scale. So to be concrete, you know, when you want to understand
05:16cancer or develop a drug, you need to be able to relate what happens at the level of molecules. The
05:23drug is a small molecule, but the impact is at the level of a full body, full organism.
05:28So part of the complexity of it is being able to collect the data and to build models that connect
05:34the different scales of biology, you know, from molecules to a metabolic pathway, to a cell, to a complex tissue.
05:43So I think this is something quite specific in biology, which in turn means that we need to develop new
05:49AI models, you know, with dedicated ways to train them. It's a bit different.
05:53So there are similar concepts to LLM and world model, but they have to be specified to be able to
06:01learn from the data we have, again, which go at different scales. I think that's the main complexity.
06:06So, Jan, is that something that has to be done from scratch, or can you kind of take some of
06:10the existing LLM paradigm, the LLM models out there, and build on that to that vision?
06:17No, it's really a different architecture. So, of course, all of this uses neural networks and deep learning, and, you
06:24know, the techniques that have been developed over the last few decades to train those systems are the same.
06:29But the architecture must be different, because the way an LLM works is that you train an LLM to predict
06:36the next word in a text, essentially.
06:38And because there is a finite number of words, it's relatively simple to – you cannot predict exactly which word
06:45will follow a particular sequence of words,
06:47but you can produce some sort of probability distribution over all possible words in your dictionary.
06:53But when you try to use the same kind of technique to train from, let's say, from video or from
06:58sensor data, from, you know, biological measurements and things like this,
07:02it doesn't really work, because there is an infinite number of plausible predictions that are all compatible with what you
07:11observe.
07:11If you ask a system to predict what's going to happen next in a video, there's basically an infinite number
07:18of plausible things that may happen.
07:19And so those generative AI architectures that are used and so successful with language really do not work for the
07:27real world.
07:27You have to come up with new architectures.
07:30The ones we use are called JEPA, that means Joint Envading Predictive Architectures.
07:34And then for, you know, certain types of applications, you need different types.
07:38But really, it's a new set of techniques.
07:41I am curious, though, to – and this is a question for you, Jan, but it applies to both of
07:45you,
07:45because both of you at one point were basically under the umbrella of some of the large companies at the
07:51forefront of it,
07:52whether we're talking about Google and Alphabet or over at Meta.
07:56Why did you feel the need that in order to see out your vision, you basically needed to do it
08:00more as a startup,
08:01rather than through what was already an existing, let's face it, you know, multi-trillion-dollar apparatus in those two
08:09companies that I just named?
08:10Jan?
08:12A number of reasons.
08:13So the first reason is that Meta, you know, felt it was kind of falling behind a little bit, the
08:22state-of-the-art in LLM.
08:23So it chose to refocus most of its efforts on catching up with the rest of the industry.
08:29And there is this phenomenon in Silicon Valley where, you know, basically everybody is working on the same thing.
08:34Everybody is trying to sort of, you know, be in the race of LLM is kind of digging the same
08:40trench.
08:41What, you know, I was working on and are still working on is different.
08:46And to some extent, LLMs were kind of sucking the oxygen out of the room.
08:52So that's one factor.
08:54The second factor is that the results we were trying to, we were starting to get were really promising.
09:00And so it was time to make the transition from, you know, academic-like research, if you want, to sort
09:07of go into high gear and make those things real.
09:11The third factor was that most of the applications of this sort of new, you know, physical AI, if you
09:17want to call it, is in industry.
09:19And this is a domain that Meta was not particularly interested in, you know, controlling industrial processes or robotics or
09:28health, for that matter.
09:32And then the third factor was that there was visibly a lot of interest from investors to really make a
09:41bet on this next revolution.
09:43Well, they've certainly made a bet on you, Jan.
09:45They're making a bet on you too, JP.
09:47And I am curious about the geography of this as well.
09:50I sit in the U.S.
09:51And, of course, you know, let's face it, us folks, us Americans, we have a certain worldview of ourselves.
09:56And there's been sort of this knock amongst U.S. investors that Europe wasn't innovating, that Europe was not the
10:02place to go if you were looking to invest in AI.
10:05Both of you are headquartered in Paris.
10:07And I am curious if that is sort of a misconception.
10:10Is there a startup culture and a technology culture that is advancing in the AI space in a way that
10:20can be commensurate, at least in terms of scale, with the United States?
10:25Yeah, I really think so.
10:27I mean, probably it started a bit later than in the U.S.
10:30But when you think of Paris, I think one of the main ingredients is the talents, right?
10:35What educated people, especially in CS, computer science, in applied mathematics.
10:40We have all of that in France, in Paris in particular.
10:44If you look at the ecosystem now, you have all the frontier AI labs, including the large players of research
10:50labs in Paris.
10:51And in the last few years, we've seen really many, many startups, very innovative startups emerging.
10:58I would say, you know, having myself live in California, for example, probably Silicon Valley is very good for velocity
11:06consumer product.
11:08But when you think of deep AI, deep tech, you know, scientific grounded AI requiring very deep expertise, in my
11:17case, you know, we need AI, but we need knowledge in biology, in medicine, et cetera.
11:21I think Paris is an extraordinary place for that.
11:24Final question, and I want both of you to kind of weigh in on this idea.
11:28It was about three and a half years ago when ChatGPT was released to the public.
11:32That was, for better or worse, kind of a watershed moment, at least in terms of the awareness of artificial
11:37intelligence and the desire to invest in it.
11:40When we talk about your world model vision of AI, when do you think we might get to that watershed
11:47moment?
11:48And what do you think it might look like?
11:50Easy question.
11:51That's a big question.
11:54We're giving ourselves a relatively long runway.
11:57We're kind of hoping we're not going to get as much time as we planned.
12:03We're quite sure we're going to have effects on practical applications with our methodology within one year or so with
12:12sort of a few select industry partners.
12:15But then if we're talking about a consumer-facing system that, you know, everyone can use, perhaps that will be
12:22driving their domestic robots, you know, we're talking three to five years roughly.
12:27So the full-fledged version of that technology, if you want, is probably three to five years away.
12:35But, you know, NLMs were used before ChatGPT.
12:40Before they came to the fore, the same type of technology was used internally at companies like Meta and Google
12:46to do things like, you know, content understanding for ranking, for content moderation, for all kinds of things, right?
12:55So the technology gets used in sort of B2B-style applications long before they become sort of a consumer item.
13:04And JP, I'll give you the last word here.
13:07Yeah, I think in biology, it's much quicker.
13:10Actually, it's already here.
13:11You know, maybe it's less visible from the general public because a lot of our use cases, applications are used
13:18in drug discovery, in biotech, in hospitals.
13:22But, for example, you know, AI models for protein design are literally used in every company today, even though they
13:29did not exist five years ago.
13:31So our cells, we just released a world model for, you know, to be able to read biopsies.
13:36When you have a biopsy, you take an image.
13:38And we have developed models that just from the image are able to reconstruct all the molecular activity inside of
13:44the cell.
13:45So this is what pharma companies need to, you know, to predict if a patient would respond or not to
13:51a particular intervention.
13:52These things are working today.
13:54So we are really at the beginning of the use of these models at scale in the industry.
Comments