- 2 years ago
Darian Shirazi, a general partner at Google's A.I.-focused Gradient Ventures, discusses ChatGPT blindspots, the risk of A.I. hallucinations, and how bigger firms should be approaching the large language model revolution.
Category
🤖
TechTranscript
00:00 Darian, thanks so much for being here with us.
00:03 You have an interesting perspective.
00:05 You know, you've been on the investment side, obviously,
00:07 also on the founder side, though,
00:09 and the bigger tech company side.
00:10 So I want to start out by just asking you,
00:13 from what you're seeing out there from your experience,
00:17 where do you think which industries,
00:19 which types of businesses are best positioned
00:23 to benefit from the kind of productivity advancements
00:27 that we're hoping to see from AI?
00:31 I think every industry is going to sort of have
00:34 some kind of impact from large language models
00:37 and artificial intelligence.
00:38 We're seeing a number of verticals
00:40 that are being impacted,
00:41 and impacted positively in terms of being automated.
00:44 The first one that comes to mind for me
00:46 is the legal tech realm.
00:48 We're seeing a lot of law firms start to adopt LLMs.
00:51 I was the first investor in Case Text,
00:53 which basically automated the process
00:55 of summarizing briefs and looking at red lines
00:58 and looking at different things within law reviews,
01:01 and Reuters bought them for $650 million this month.
01:05 So we're seeing a lot of disruption in legal tech.
01:07 We just led a series A in a company called FileRead,
01:10 which is automating e-discovery with large language models,
01:13 and the team came from Caltech.
01:15 They sort of built the technology themselves
01:17 and now have large contracts with law firms
01:20 that are looking to have an edge in litigation.
01:22 So I think that's one vertical that I would say
01:25 is the first that's already experiencing
01:28 a strong sort of innovative lever behind them
01:32 and seeing growth from LLMs.
01:34 And is there anything specific to that industry
01:37 that kind of makes the needs and the use cases
01:41 very clear in the ROI there,
01:44 but also the kind of cultural acceptance of it?
01:47 Are you seeing that there, too?
01:49 Yeah, legal tech has historically been a difficult place
01:53 for venture-backed companies to succeed.
01:55 So it's kind of a role reversal where we're seeing that
01:58 as a place where there's already been a big liquidity event.
02:01 There continues to be a lot of active investment.
02:03 I think that comes down to the fact that the majority
02:05 of the content that lawyers deal with is text-based,
02:08 and so these large language models are particularly good
02:12 at summarizing information that lawyers
02:14 necessarily would have to read through extensively.
02:17 So I think the technology is very well suited
02:20 for that vertical to start.
02:22 But there are also others.
02:23 So healthcare, I think, is another one.
02:25 There are a number of companies that are starting
02:27 to do summarization of insurance data
02:30 to understand whether a specific diagnosis code
02:32 is actually going to be mapped to a real insurance plan
02:35 and what the cost breakdown would be.
02:37 That usually has to be done with manual labor,
02:40 reading through the insurance materials
02:42 and then assessing what the insurance breakdown would be.
02:45 Sometimes that leads to a dispute.
02:47 We're also seeing in a lot of other areas,
02:49 such as computer vision.
02:50 I spoke to a company today that is now using image models
02:54 to basically detect whether a skier has fallen
02:57 from a ski lift or not when they're about to get off
03:00 of a ski lift so they don't need a lifty
03:01 at the top of the lift.
03:03 It's kind of a weird use case, but it gives you a sense
03:05 as to the level of detail that some of these companies
03:08 are going to that the ski industry is suddenly starting
03:11 to adopt image models.
03:13 And the company is selling a generic LLM testing platform,
03:16 but one of their first customers happens
03:18 to be a ski lift company.
03:20 So I thought that was kind of interesting to see
03:22 that even these long tail weird use cases
03:25 are already seeing adoption from these large language models,
03:28 image models, et cetera.
03:30 The more obvious ones to us, I think,
03:33 are general productivity.
03:35 We're seeing a lot of companies, such as Notion,
03:37 which is an incredible platform.
03:39 They've integrated AI, and a lot of their customers
03:42 are now using AI to generate content and automate
03:45 and speed up their work.
03:46 We're seeing Google adopt it, Microsoft, et cetera,
03:49 general productivity will be obviously impacted positively
03:52 in many ways by LLMs.
03:55 But I think that legal and health tech
03:57 are going to be two areas that we wouldn't expect,
04:00 and it's only the beginning.
04:02 Every single software company is going to have to change
04:05 their mindset because we're no longer buying these solutions
04:08 that sort of allow you to put things in the cloud.
04:11 We're also--people are going to want to buy solutions
04:13 that are automatically intelligent,
04:15 that can give them the correct answer,
04:17 and that's going to disrupt and change every single vertical.
04:20 As a result of that will be significantly more jobs,
04:23 I think.
04:25 I don't think it's going to take away jobs.
04:27 I think it's going to make it such that there will be more
04:29 of a need for training data to make these applications
04:31 even more intelligent over time.
04:33 Unless you're working a ski lift.
04:35 Maybe the liftees will have more time to ski,
04:38 but I thought that use case was particularly funny
04:40 and interesting, so I wanted to bring it up.
04:42 Very interesting.
04:44 Not where your mind goes to right away when you think
04:46 about AI or AI more broadly.
04:48 I want to ask you about blind spots.
04:51 What do you think are going to be some of the biggest blind spots
04:55 and the biggest hurdles for companies?
04:57 You know, you're talking about industries
04:59 with the legal profession, with health care
05:03 that aren't necessarily known as early adopters
05:06 of new technologies, so how are big companies looking
05:09 at actually deploying some of this stuff?
05:12 Yeah, I think that there are a couple challenges.
05:15 One is citations, is that currently we really don't have
05:19 any sense within chat GPT or GPT 3.5 and 4
05:23 where the content is coming from,
05:25 where the specific answer is actually coming from,
05:28 and that is a very complicated project,
05:30 one that I know that is being worked on by a lot
05:33 of other AI researchers and something that I think
05:35 is going to have to be resolved, because ultimately,
05:38 if a large language model learns from this interview,
05:41 Fortune should get credit for it, and I should get credit
05:44 for whatever I've said as well, I believe.
05:46 Unfortunately, the current models don't allow for that,
05:49 and we have no idea where it actually learned
05:51 the specific answer.
05:52 So that's one where I think we're going to need
05:54 to see a lot of innovation.
05:56 I think second blind spot is definitely data privacy,
06:00 is that currently large language models are learning information
06:03 that may be personally identifiable.
06:05 We need to figure out how to filter that information out.
06:08 We're seeing a lot of pitches these days of companies
06:10 that are helping with that process to make sure
06:12 the training data isn't training data that can't be used
06:15 in a large language model.
06:17 I also think that answers being incorrect,
06:20 being able to tune these models and tune the responses
06:23 to say, "Hey, this is incorrect or correct."
06:26 Ultimately, if we have a health care company
06:29 that's based on LLMs and it's helping a surgeon
06:31 when they're in the middle of surgery,
06:33 and the answer is incorrect, that could have
06:36 a really bad outcome.
06:37 We can't have those kinds of outcomes.
06:39 And so those are really three areas that I think
06:42 are big blind spots that need to have significant research done
06:45 before a lot of the applications that will emerge
06:48 over the next 10 to 20 years are actually viable.
06:51 You don't want any hallucinations for health care applications,
06:55 I'm guessing. That sounds--
06:56 - Exactly. - Yeah.
06:59 Do you have any advice for, again,
07:02 just going back to some of the bigger companies
07:04 that are testing it out, if not fully deploying
07:08 some of these AI-powered applications,
07:11 how not to just fall into some of the hype,
07:15 not to--you know, in some of this,
07:16 you don't want to just throw spaghetti at the wall, right?
07:19 You want to get it right. There's a lot at stake.
07:21 What's your advice on that front?
07:24 I think the key thing is to realize that there's going to be
07:28 many different steps along this journey,
07:31 the same way that there have been in a lot of technology super cycles.
07:35 And I think in the case of the large language model community,
07:38 companies should really start to adopt different models
07:42 but not be completely invested in one ecosystem yet.
07:45 They should really think about, "Okay, let me test OpenAI.
07:48 Let me test all the different models that are out there
07:51 before making a fully-baked decision as to,
07:53 'I'm going to base my whole next generation of my business on this.'"
07:57 And I think that there are many different platforms
07:59 that are going to help with the process of testing and tuning
08:02 and ensuring that they actually pick the right models
08:05 for the right application.
08:07 So I think it's definitely early days,
08:10 but there are a few key things that can really improve
08:13 the experience for customers across different industries right away.
08:18 I think legal tech is one.
08:19 We also invested in a company called Range,
08:22 which is in the wealth management space that allows for people--
08:25 that has automatic advice on where to invest money
08:28 based on market-changing conditions
08:30 and allows people to ask questions about their money.
08:32 There's definitely sectors or areas that are going to benefit immediately,
08:36 and I think it's just take it step by step
08:38 and not go all in on a platform just yet
08:41 until there's been more development in this entire ecosystem.
08:44 So speaking of some of your investments,
08:47 I'm sure that you are getting all sorts of pitches
08:50 from all sorts of companies that have tacked on AI
08:53 to their name and description.
08:57 How are you sifting through all of these different investing opportunities,
09:02 and what do you look for as an AI investor?
09:06 Yeah, it's busy.
09:08 I mean, today I saw eight new companies,
09:10 and that's like probably the average day.
09:13 I would say that everyone's trying to build something
09:16 based on these large language models
09:17 or building their own foundational models
09:19 or customizing for different industries and applications.
09:22 It's difficult.
09:23 We have a relatively small team,
09:25 and we're very specific about the people that we bring on
09:28 because you have to have the right blend of technical skill set
09:31 as well as business mindset
09:33 in order to identify the right opportunity.
09:37 I have a rule set that I usually follow for founders.
09:40 I really want to make sure that I can trust the founder
09:44 and that I can trust them probably for the next 10 years
09:47 because ultimately that's the journey that we're signing up for.
09:49 When we invest in a company,
09:51 we expect to be with that company for 10 years or maybe longer.
09:55 If you look at every major IPO that's out there,
09:57 that's how long it really took from founding to exit.
10:00 I think that second is I'd really like to know
10:02 that they're going to be resilient.
10:04 There's going to be many different waves.
10:05 There are going to be different phases
10:07 within this AI renaissance, so to speak.
10:10 And we need to make sure that founders
10:12 are willing to sort of roll with the punches.
10:14 And if there happens to be sort of a disillusionment
10:16 around AI in the next couple of years,
10:18 we need to know that they're going to dig in
10:19 and really build the right company
10:21 and continue to execute on their vision and plan.
10:24 And crisis is common in startups.
10:27 You have to be able to survive
10:29 through all of the different crises that come your way,
10:31 whether they're people-related or technology-related
10:34 or customer-related.
10:35 There's just so many different challenges
10:37 that founders are going to experience.
10:39 And then I think the third is,
10:41 do I really believe that the founding team can learn
10:44 and learn quickly over time?
10:46 And as they sort of evolve the company,
10:49 go from being this project
10:50 where they're looking to find product-market fit
10:52 and then becoming a company where they're monetizing
10:55 and working with customers
10:57 and then ultimately scaling the business,
10:59 are they sort of the founders
11:02 with the right level of neuroplasticity
11:04 to survive all three phases is a really big deal for me
11:08 because I don't want to be in the business
11:09 of finding a professional CEO for a company.
11:12 I want to work with the founders
11:14 from the beginning to the end.
11:15 And so I really look for that in the early stages.
11:18 So far, I've been pretty good
11:19 at finding these different characteristics,
11:21 and I think the rest of our team has as well.
11:24 And we just continue to look for those things
11:26 and quickly screen for it.
11:28 As far as technical viability,
11:30 those are things that are sort of innate in our team.
11:33 We understand what is possible and not possible in AI,
11:36 and I think that's what's given us a leg up
11:38 in a lot of these conversations with portfolio companies
11:40 vis-a-vis other investors
11:42 that maybe haven't been investing in AI
11:44 for the last six years.
11:46 Okay. So last question for you.
11:48 I know most of the companies you're talking to
11:51 are probably not trying to do what open AI is doing
11:54 and they're not trying to build from scratch
11:56 these large-language models,
11:57 but a lot of these companies probably still require
12:01 quite a bit of capital, compute power,
12:03 all of that, right? A lot of resources.
12:06 Do you think that in this particular case
12:09 with generative AI,
12:11 that bigger companies could have the upper hand
12:13 versus the startups
12:15 who have historically been the innovators?
12:17 How do you look at that?
12:19 Well, I think it used to be the case for general cloud,
12:23 but in the AI era, I think that's actually changing.
12:26 I mean, NVIDIA has become the kingmaker
12:28 in terms of allocating GPUs.
12:31 They've allowed companies--
12:33 they're allocating GPUs to companies like CoreWeave
12:35 and Lambda Labs.
12:37 Lambda's one of our investments,
12:38 and they really do--or they really are trying
12:40 to spread the amount of compute power
12:42 across all these different companies.
12:44 I actually think that software innovations
12:46 are going to allow for compute
12:47 to be faster and cheaper over time.
12:49 So I would say that in the AI era,
12:52 there's actually more democratization
12:54 than there was in the previous cloud era,
12:56 and that's been promising,
12:58 especially as a venture investor.
13:00 So I would take the opposite of that,
13:02 that I think that this is definitely going to be
13:04 an area where another, you know, trillion-dollar company
13:08 or multiple trillion-dollar companies
13:10 will emerge because they're able to access
13:13 different types of resources from NVIDIA
13:16 and that there's more democratization
13:17 of the compute power over time.
13:19 Okay, fascinating stuff.
13:21 Darian, thank you so much for being with us.
13:24 Absolutely. Thank you for having me.
13:26 Thank you.
13:27 [BLANK_AUDIO]
Comments