OpenAI Targets Custom Silicon in Broadcom Deal

Watch OpenAI Targets Custom Silicon in Broadcom Deal - Bloomberg on Dailymotion

Transcript

00:00What's been so interesting about these golden touches from OpenAI for some of the big publicly

00:04traded companies is usually OpenAI has been given a sweetener by NVIDIA or by AMD. Broadcom,

00:10it just seems to be independence that they give to OpenAI. Yeah, and I think the model here is

00:15the Google GPU model. I mean, when you think about Broadcom's almost $20 billion run rate for AI

00:21chips, more than half of that is from Google GPUs. So what OpenAI is saying is you help us get there

00:29in terms of the ramp up like Google TPUs, which is in their seventh version of chips. I mean,

00:36and they've done it at a very quick pace. So from that perspective, it will help OpenAI reduce costs

00:43of up to 30 to 40 percent per gigawatt. If you think about, you know, one gigawatt takes 40 to 50

00:49billion. One gigawatt with Broadcom chips will be at least 30 to 40 percent cheaper because the cost

00:57of those chips is the highest component in that gigawatt buildout. So Broadcom helps you lower

01:03that cost of chips. And I think that's the model here that, yes, we want merchant silicon,

01:09but we also want custom silicon with Broadcom because that's the kind of diversification Google

01:14has. And that's why their cost of infrastructure is the lowest among all the hyperscalers.

01:19Well, do the read across for the TPUs from Google. Who else is in the mix there? Because here Broadcom

01:25says, I'll help you custom design a chip. I'll help you with networking gear. But there's a lot

01:29more to an AI data center than all of that. Absolutely. And you need to source the power.

01:34You need all the other deals. But in the end, when you look at, you know, what Amazon is trying to do

01:39with Tranium, they're doing that with Marvel. Microsoft is doing that with Marvel as well.

01:45And they haven't had the same kind of success that Google has had with TPUs, with Broadcom.

01:51So to my mind, you know, it was natural for OpenAI to try with Broadcom, given the success that,

01:57again, Google has had compared to everyone else who is trying to do custom silicon.

02:02And yeah, they will do deals for power. That's what OpenAI is good at in terms of sourcing different

02:07providers. That's what Sam Altman has shown. But clearly, chips is the component that costs 60 to 70%

02:14of the data center. So you want to make sure you get that at the lowest cost. You won't be able to do

02:20that with NVIDIA. NVIDIA will still be the highest cost chip provider, even though they are making an

02:25investment. AMD will likely cut its cost, but it won't be the same performance per watt. Broadcom will do it

02:33custom specs for you, and then they can do it at scale that Google is doing.

02:37And it's for inference. And I'm interested if you can interpret when OpenAI, Sam and Hocktan get

02:43together on a podcast and announce this sort of a deal. What is it that by understanding your own

02:48large language model and the needs of it that can really be built into the custom silicon?

02:52I mean, just this past weekend, I read about tiny recursive models. So everyone is looking at how

02:59these large models can be run more efficiently in terms of inferencing costs. And you know, whether it's

03:05tiny recursive models or some other form, you want minimum latency as well as, you know, power is your

03:11real constraint. So you want maximum performance per watt. So if you're optimizing for those two,

03:18you are going to go with custom silicon because that's what Google has shown us. They can run YouTube

03:23videos best because it's their custom silicon. No other merchant silicon can give you that kind of

03:29performance. And I think that's what OpenAI is going after.