- 6 minutes ago
TK
Category
😹
FunTranscript
00:07all right Karina well I was glancing at your resume and it's a little bit
00:13impressive so you graduated from MIT with a double major in math and physics
00:17then you went to Oxford to be a Rhodes Scholar then you went to Stanford to
00:22earn not just a PhD but also a JD I barely passed high school algebra but
00:30then yeah but then you dropped out so I'm not even sure what my question is it's
00:34just how did you do all this at 24 years old I mean I think it's like a typical
00:43journey of a math Olympia kid as you go to like kind of MIT and the more math
00:48competitions like try to do undergrad research and really your ambitions to
00:54go to grad school and be a math PhD and hopefully you can continue in math
00:58academia and prove some of the very hard math problems that generations of
01:04mathematicians were puzzled by I think the most interesting sort of like
01:08intellectual unlock for me is that we're in an era of AI and why don't just you
01:14know build an AI mathematician so if you think about like the lifetime of
01:18mathematician and probably two really really hard problems would be I think a
01:23good life and with AI was you know a billion AI mathematician agents running
01:29every day I think we can encounter like a lot more interesting mathematical
01:34phenomena and you know really just bring AI for math and AI for scientific
01:38discovery to a level that we didn't imagine before that's why I dropped out yeah and I
01:43think it's worth telling people that your company is less than a year old
01:47already 1.6 billion dollar valuation 20 employees you're in the offices where
01:53Facebook first was and in December it scored a perfect 12 out of 12 on the
02:00Putnam competition just for people who don't know what that is which was
02:04definitely me before I started looking into this explain the significance of that
02:08yeah so putnam exam is really hard I think it's like notorious so that people
02:13generally say the median score is zero so out of like 120 scores you could
02:18possibly score and each problem has I think like 10 scores so if you like get
02:23one out of the 10 steps correct you will get at least score one more than 50% of
02:28math majors selected from all the amazing universities got a zero and I think this
02:34is usually kind of like a humbling moment for you and the kids who are really bright
02:39you know like doing math competition all along and people just did not like expect
02:45AI to get a perfect score on this one so the the high school level of math
02:49competition called the International Mass Olympiad you know many teams open AI
02:54deep mind all attempted it and no one got a perfect score I mean they got the
02:59borderline for gold medal and that was like an incredible accomplishment so putnam
03:04on being actually harder putnam 2025 is deemed as harder than the IMO people just
03:10didn't expect that there will be an AI mathematician that got the six perfect
03:16score in human history so five humans have got perfect score over a century of
03:21putnam exam so I think that was that was quite interesting like I definitely did not
03:25even score 40 so and I got some award for it so it's very interesting to see to see an
03:32AI just a kind of like punch me in the
03:34face it's like all these years of math training you know like it's very
03:37interesting moment and so you're you develop AI that it is very good and good
03:43at doing math but what do you actually want to do with that yeah so there is this
03:51sort of I guess two axis one axis is super intelligence we're at a time where I
03:57think if you're in the sort of model layer you really want to go for ASI
04:02specialized super intelligence that's one goal that's can we have AI not
04:08replicate but actually exceed some of the human genesis that ever existed in
04:14history so if you think about like you know we are in a very reasoning scars like
04:19society in the sense that after Galois died from the duo like so for I think
04:24decades group theory was like setback and Ramanujan died and elementary all that sort of
04:30you know extremely fundamental functions formulas properties were not discovered
04:37for the next couple decades so it's like humanity has been genius bound and if you
04:41want to build an AI super intelligent mathematician you can potentially have
04:46like just like really orders of magnitude more mass discoveries and that unlock is
04:53hopefully going to sort of fuse into like various like applied science domains and you would imagine
05:00the collaboration between mathematicians physicists mathematicians and molecular
05:04scientists mathematicians and computer scientists neuroscientists like is to be a lot more because you know
05:11previously the the top you know 0.001 percent of mathematicians kind of stay in
05:15academia stay in their pure theoretical field they don't necessarily collaborate with
05:19supply scientists so we understand very little about the universe we also understand very
05:23little about the human brain and you know for once in a decade some physicists
05:28usually study string theory in PhD and realize I wasn't going anywhere enter I don't know
05:32theoretical computational neuroscience and then make a lot of interesting discoveries I
05:37know I actually study with a bunch when I was in a UCL Gatsby and it's like wow like that
05:42amazing you know capability of mass
05:44reasoning like applied to neuroscience was able to corroborate with a lot of
05:48experimental you know neuroscience finding so that's their act that axis of like
05:53super intelligent there's the other axis which is I think I tend to say that we
05:58are in an era of I think unfortunately Schrodinger's super intelligent so if I
06:03have an LM that's going to tell me the mystery of the universe and every five times I
06:09prompted actually there's one that's correct and I don't actually know which one is so we
06:14have like a million lines of math proof generated and is any human actually going
06:19to dig into the details and kind of like see you know where there might be a
06:23mistake so I would say that super intelligence is at its sort of max power
06:27when it's verified when you know that you can just execute the output like a
06:32computer program and go I mean you can get sort of the verifiable signal which is
06:36what we're doing so all the math solutions the pen and exam did not need to be
06:40cross-checked by human experts they are written in lean which is a computer
06:44program for math proofs so once you run that computer program and then it's
06:49successfully compiled under the either safe verify or various sort of lean
06:54community standard you know that it is it is correct so it is actually you know all
06:58the possible all the problems you know must be either 10 or there and there's no
07:02sort of partial partial scoring so that's that's another axis which is if you kind of
07:08generalize from there you can formally verify code and code could mean software
07:12code and could code could also mean hardware code I think that taking the AI
07:17mathematician technology into the domain to change how hardware verification is
07:22typically done to kind of make software verification a reality are some of our
07:27near-term ambitions but in math of course there is a right answer yeah with code
07:32code there there's not it's more of a art than a science so how do you use math to
07:38verify it yeah so if you tell me what property you would like your computer
07:43program to satisfy if you specify those things for me you're almost like giving me
07:48the mathematical statements for me to prove so as long as you are willing to
07:52specify things we can make an attempt to prove it actually you can make it a lot more
07:58concrete in terms of like what is safe what is good code I think a lot of the
08:03you know need for software verification has long existed I mean since like last
08:08century in various like mission critical domains specifically the European Space
08:13Agency Arians project I mean the spacecraft was around the time of the
08:17Challenger people see a need to do formal verification a lot of these formal
08:21verification were done by humans traditionally just because you don't have you
08:26know any other way and these formal verification experts you know currently
08:31it's getting like less and less of the population because there's a big sort of
08:35formal verification push around 1980s and around also like 1990s actually the Paris
08:41subway system the automatic switching that was also formally verified the trade
08:46union demanded that you need to formally verify otherwise people won't be safe so
08:50trade union for technology yeah so it actually has a long history and if you
08:55think AI has changed some of the sort of front-end software engineers life it
08:58has not really changed sort of like large distributed system especially systems
09:03where like safety becomes like a non-negotiable I think that for AI to
09:07change some of these experts life is something that we're working toward and you
09:12last year hired Ken Ono who you described as you know a a an idol to many mass
09:20students including you the Walter journal described at the time he's leaving his
09:24tenured job to work for a 24 year old so what's it like having someone you know
09:30twice your age your mentor work for you do you guys ever disagree and what advice has
09:37he given you yeah it's funny you ask that I know Ken since I was a freshman applied to
09:42his RUU research experience for undergrads and it's one of the countries like most
09:47prestigious RUU I was actually put on the waitlist so he did not consider me smart enough to work in
09:52his RUU and and it was very interesting I mean a whole summer I tried to be as
09:57productive hard-working as I can and I think there are like two or three papers
10:01coming out of it now we're at a scenario where at 10 a.m. some math professor friend of
10:07Ken will send us a research problem like an open conjecture that no one has proven
10:11before and that professor or his grad students could not make progress on it and by 2 p.m.
10:16max improver like generated like thousands of lines slim proof file back and that
10:22professor get like shocked that's kind of like it's on a daily basis right so if
10:27you think about how much sort of the reasoning capability has gained and what
10:32that means for mathematicians it's actually not a competition it's something that's
10:36quite refreshing to see is that you can scale your impact you can actually have a
10:42chance of finding out the mystery of all the math problems that you care about and
10:46that just was never the case I mean especially with kind of federal funding
10:50cuts and NSF and all the math departments people are finding you know there are less
10:55headcounts for grad students and for the people to really try to explore all the
11:00curiosity and I think humanities like one of our fundamental need is curiosity so
11:05even at a time if AI is able to be a superhuman mathematicians I still think
11:08mathematical activity will also be reading understanding the proof find out if
11:13there are like new problems that can arise from these like solved problems and be
11:17able to build even grander theories and you know have more conjectures and have
11:21these like AI mathematician automatically close those conjectures so in the future I
11:26think math discovery is going to be more collaborative it's like going to be like
11:31github large-scale project here I can you can toss like maybe even the lung lens
11:35program into many many different pieces and for the AI mathematicians to make
11:39progress on all the tiny pieces and then maybe mathematicians will be functioning
11:42on a totally different abstraction level to be kind of overseeing you know how
11:47these progress come together and I think that's extremely exciting I actually think
11:50that's why Ken joins because he sees a new way of doing mathematics and I think
11:54some of his peers are also slowly realizing that and also sending us a lot of
11:58requests for proofs for all the formalization which is to convert a math paper
12:02they just recently wrote into lean because digitalizing all modern mathematics is
12:08actually another ambition that a lot of them you know frontier mathematicians do
12:11have and do you guys ever disagree on anything we disagree on what problems to
12:15pick so I mean I'm a number theories he's also a number theory there are certain
12:20problems that I kind of like grew up and dreaming about like I just want to like
12:24push the you know gap from a certain number to a certain number and then can we
12:30think well that's just too hard like I mean this is he's an expert he knows the
12:36mechanism of how to you know approach something like that but then I always
12:40disagree and be like well you never know because we were surprised by our
12:43technical capability again and again and you know you are one of a lot of neo labs
12:48which we write about quite a bit they're kind of you know this phenomenon right
12:52now startups valued at over a billion dollars that don't have a product a lot of
12:59people see this as proof of an AI bubble what do you think about that yes I think the
13:04definition of neo lab is like something like you know deep focus on research which
13:08we do have and lack of commercialization plan I actually think that in our case
13:13the transferability of like math they're improving formal they're improving into
13:18formal verification of code such as on the famous benchmark code very now we
13:23actually saturated it I mean also the transferability to say hardware
13:28verification we have been proving a lot of the industrial partners like you know
13:33circuit designs and those are cases where they say no other formal checker
13:38could do so I think we actually have a pretty clear kind of commercialization
13:42goals that we are working toward so which make us you know disqualify for one of
13:47those like fancy new labs list but what about the rest of them what about the rest of
13:51them I think there are a lot of very very interesting kind of directions that the
13:54new labs are pursuing specifically like me personally I think personal
13:59intelligence like continual learning that's something I'm quite bullish about I'm
14:04actually less bullish about this sort of energy-based model approach so I think
14:09new labs in that category will find it a bit hard to explore both a fundamental
14:14technological shift if it succeeds and also kind of what the business
14:19implications are of course I can be proven wrong I think new labs in terms of AI for
14:25science in general like life sciences or molecular you know material sciences and
14:32you know other other kind of like science labs I would say is something that I
14:37think is have quite a lot of promises and are usually actually undervalued and you
14:43know another thing we write about all the time is the talent wars with you know
14:46the open AI's and metas offering hundreds of millions of dollars you have to
14:50compete against them you have to compete against the neo labs yeah how do you do
14:54that I'm sure the one point evaluation helps but I don't know I think so
14:59perhaps we had a little bit of a head start we're actually one of the very early
15:03once after SSI and thinking machine knows probably not a lot of new labs around
15:07back then we had really amazing talents coming from big tech their industry
15:13veterans they are you know a bit tired of the PSC culture or all the politics at
15:18big shops and they just want to like work at a place where they feel like you
15:24know it's not sort of like job security because it is a startup there's no job
15:29insecurity but more like security for the research direction like when you're in a
15:34big lab and your research direction changes on like monthly if not weekly basis
15:39that's actually another sort of security they are pursuing right like you want to
15:43work at a place where you know that what you're working on still has impact in
15:46like two months right and then like for us where we are very clear in declaring
15:52that the company's DNA will be mass and always be mass we're not pivoting we're
15:57not shifting we could generalize into the you know few commercial areas that we
16:01find promising and you know talk to people in those fields that we consider you
16:07know pretty relevant to apply the technology into but the fundamental does not
16:11change and I think that's something quite appealing it's like we're not gonna rest
16:15until we get to the moon which is I think the definition of a moon shot
16:18startup or a new lab yeah which is very topical all right well we're gonna finish
16:22with a lightning round okay so get ready everyone god all right what job do you
16:27think will be obsolete first coders or mathematicians
16:35like completely obsolete I I'm gonna say mathematicians okay yeah which startup
16:45founder do you most admire we start a lot of them like Liam from periodic class Liam
16:51fetus Liam fetus yeah Eric Zelligman from humans and Andrew Dye from Elorian which big
16:59tech CEO do you most admire Demis has have this deep mind I actually do admire
17:07like Elon that actually I think the changes he makes are necessary which
17:13startup do you think is most underrated which startup I think so our actually
17:22upstairs neighbor parallel they are in the search space I think both parallel and
17:26exa building sort of search for AI and agents it's something that is going to
17:32be a huge category okay and which startup do you think is most overrated
17:38oh god most overrated I I think our competitor
17:53that's harmonic we should say okay but but they were founded in 2023 yeah kind of
17:59last year you already are valued at more than them but they are backed by Nvidia so
18:04you got to get those I try to be grind grounded yeah okay all right what's more
18:09valuable in a year open AI or anthropic tied okay yeah okay okay yeah trying to
18:18play safe here yeah smart all right which company is the one that tries to poach most of
18:25your researchers thinking machine okay okay and was math invented or discovered
18:37Jesus invented upon a certain minimum threshold so people can start discovery okay yeah if you had to
18:46pick a number to represent your personality what would it be and why for everyone says in Chinese
18:53culture 4 is extremely unlucky so I like to be an underdog you know I want to be yeah 4
19:00and it's a
19:00square so yeah and I have to say chat GPT came up with that question just to be honest there
19:08all right finally what's a hot take or prediction for the year ahead that would surprise most people in
19:14our audience oh predictions a year ahead I think that software verification is going to play a more
19:27mainstream role in the AGI timeline so in other words it's not to be thought of as an infrastructure
19:36that's complementary to say you know the gen AI in terms of you know like it cleans up people's mistakes
19:44mistakes or sort of like solves hallucination I think the fundamental promise of software verification
19:50is you have infinite number of agents being able to produce verified output pass it on to each other
19:57and just keep building from there all the code ship directly to deployment and I think that's verified
20:04super intelligence which is going to compound and accelerate the entire AGI timeline so it's not to be
20:10thought of it's something that fixes mistakes but something that is going to like make the upside of
20:15super intelligence a thousand times larger if not more I don't know if that qualifies as a hot take but
20:21at
20:21least a technical one okay all right well thank you so much thank you appreciate it thanks
20:25you
Comments