- 2 days ago
IBM Fellow and Chief Technology Officer of Systems Development Christian Jacobi joins WIRED to answer the internet’s burning questions about microchips.
Category
🤖
TechTranscript
00:00I'm Christian Jacobi, Chief Technology Officer for Systems Design at IBM.
00:04Let's answer your questions from the internet. This is Microchip Support.
00:12From the explain like I'm five subreddit, how are microchips programmed to know what is a one
00:17and zero? So let's step back and talk about why zeros and ones are so important in computing.
00:22You can really encode all sorts of information as a long series of zeros and ones. Take the ASCII
00:28character set, for example. It enumerates all the characters, the letters, special signs,
00:35in a long list and then it assigns a combination of zeros and ones to represent each of these letters.
00:40And then you can take a whole book, for example, and just string all the letters and characters
00:45and spaces and commas and exclamation marks to create a long sequence of zeros and ones.
00:50In a computer chip, a zero is usually represented by no voltage and a one is represented by some
00:56voltage like one volt or 1.5 volts. Transistors in a computer chip can then modify the signals
01:02by either switching a transistor on or off and performing certain computations. You can build
01:08specific circuits using transistors like adders or multipliers, but you can also make it programmable
01:13so that you get software to run and perform ever more complex operations on these strings of data
01:19that you're sending into the computer chips. Rufflecopter4 asks, just how on earth does a
01:25transistor physically work? Well, the easiest way to think of it is like a valve. You have an input and
01:30an output and a valve handle. In electronics, with the transistor, we call the input and output source
01:35and drain and the handle is called a gate. An electrical signal connected to the gate opens up the channel
01:41between the source and the drain or it keeps it closed so that no electricity can flow. On a modern
01:47chip like
01:47this, there are billions of transistors and of course they don't switch like very slowly like a handle,
01:53but they can switch billions of times per second. Eat beef now. Why there are only few chip makers in
01:59the world? Well, the modern chips are designed in very complicated processes. Like we're down to
02:055, 4, 2 nanometer design points and the manufacturing is extremely complicated. Building fabs that can
02:13manufacture chips like that is extremely costly and driving the development work to get to the next
02:19technology node from say 3 to 2 and from 2 to 1.4 is an extremely costly undertaking as well.
02:26So that's
02:26why we've seen a big consolidation over the last 10 or 15 years and we only have a few companies
02:31who can
02:31really afford to do the development work and build those fabs. So some of the big manufacturers today are
02:37TSMC in Taiwan, Samsung in Korea and Intel here in the US. All these companies are also building
02:44fabs in the US and other places in the world, but that's where they originate.
02:49Venti9 asks, why do computers get slow with time? Let me bust that myth a little bit. When you own
02:54a
02:54computer over a period of time, you're loading more and more programs onto the computer, you're getting
02:59firmware and software updates, you're loading more and more data on it. So it's not that the hardware gets
03:04slower. You're just asking the hardware to do more because you're kind of accumulating a lot of junk
03:09on the device. It's not like the chips wear out and the hardware gets slower. It's just you're asking
03:13more of it. From Hatchface, stupid question. Why do they need so many new data centers anyway? Not a stupid
03:21question. With what's going on in AI over the last few years, we've really seen an explosion in new data
03:27center construction. And so let's just step back. AI has had some really big breakthroughs over the last
03:33three, four, five years. And it's really driving worldwide productivity for knowledge workers.
03:41Now, we spent trillions and trillions of dollars in wages for knowledge workers on the globe. And if
03:47it can make knowledge workers more productive by only a few percent, that is a massive market,
03:54a hugely valuable market, many trillions of dollars worth. And so you see a lot of companies building
04:00data centers to capture their share of that market. Now, these data centers are really complex. They
04:06need enormous power supplies because they get filled with computers for storage, general purpose computing,
04:14and then of course, a lot of GPUs for all the AI processing. So while these data centers are really
04:20huge infrastructure projects with like big buildings and power supplies and cooling inside of the data
04:27centers, ultimately, there are millions of chips, memory chips, storage chips, general purpose processors,
04:34and of course, lots of GPUs for the AI processing. Since Leda is asking, what are all those billions of
04:41transistors in my CPU actually doing? So transistors are really microscopic. A modern day
04:46transistor has only a few nanometers of size. A human hair is 100,000 nanometers in width. So when we're
04:53talking about five nanometers, that's like a tiny, tiny, tiny fraction of the width of a human hair. These
04:59billions of transistors, each one of them is a tiny switch. And we can form gates, AND gates, OR gates
05:06that
05:06combine two or three or four signals and compute the AND of all these signals or the OR of all
05:11these
05:11signals, right? All four of them one, then an output is one or is one of them not a one,
05:17then the output
05:18would be zero of the AND gate. Then we take these gates and form more and more complex circuits. We
05:23can
05:23build small adders, we can build multipliers, we can perform those computations in a loop. We can then add
05:29program functions so that you can actually program the chips and perform ever more complex computations.
05:34And so because we're putting all of these circuits and multiple cores and a lot of memory on these
05:39chips, it adds up to billions of transistors. Minoshi asks, how can chips have billions of
05:45transistors but have very few external wires connecting them? It really matters how much data
05:50you need to get in and out of the chip versus how much computation you perform on the chip with
05:55each
05:55piece of data. Take this chip for example here. These are all the connections on the backside. They are
06:00combining the power supplies for the chip as well as the input and output signals. And then there are
06:05billions of transistors and a lot of memory on this chip to perform the actual computations.
06:11Dickhead and L is asking, why do computer chips warm up? The computer chips consist of transistors
06:16and every time they do a switch, there's a tiny current flowing through the transistor and the metal
06:21stack. And when that current flows, a few electrons get moved around and they push against the atomic
06:26structure of the metal. And that creates friction, almost as if your hands are rubbing together.
06:31That friction is causing the heat in the chips. From the Ask Science subreddit, if transistors are
06:38so small, like a few atoms, then how do we build them and put all of them on a CPU?
06:43We start in the
06:44manufacturing process with a blank wafer. And then we put some photoresistive material on the wafer. We
06:49coat the whole wafer with that. And then a mask that has all of the fine structures of the design
06:55is used to
06:56shine a light on the photoresist. And then we edge out the areas that have been blocked from the light.
07:03And we can deposit metals or we can dope the silicon and create the semiconducting properties. And then
07:09this happens in many, many, many layers. First, we build what's called the front end, which are the
07:14transistors using repeated steps of photoresist, shine light on it, edge it, deposit. And then after the
07:23transistors are done with multiple layers, we then put the metal stack on top, which connects all of
07:28the transistors. Nowadays, with the fine structures that we have, the two nanometer transistors,
07:33it actually matters what kind of light we're using for that imaging. We're today using extreme
07:39ultraviolet light because the wavelength of the light itself has to be small enough to even be able
07:46to show the fine structures that we need on these chips. The machines that do all that work are
07:52massive, like the size of an entire room, because they need to super precisely position the wafer.
08:00They need to position the mask. They need to have the laser light position. And all of that needs to
08:05be
08:05like really in lockstep to be able to create these super fine structures of nanometer size. Then there
08:12are super fine machines that can cut the wafer into individual chips. And we call that dicing. And so you
08:19get these chips and then these individual chips are put on what we call a module. That's the little green
08:26board with two chips. And then there's a metal stack inside here that interconnects the two chips on this
08:31module, as well as connects them to the underside, where we have the pins that are driving the inputs
08:37and outputs of this chip. A Reddit user asks, how was the first computer chip created with no computers to
08:43create it? Well, really, the first computers were designed by hand on a piece of paper. The circuits
08:48were drawn out on a piece of paper. And then people would connect the different components with little
08:54wires that they would solder to the different components. I myself, and I was at Saarland University in
08:59Germany, was in a computer class where we had what were called wire wrap boards. You would put components
09:05into the wire wrap board on one side, and you would connect little wires on the backside to interconnect
09:10all these components. And we built a small calculator using that primitive technique. But in the 70s,
09:16whole computers were built with these wire wrap boards. Nowadays, of course, we have very powerful
09:20computers, and we can use these powerful computers to build ever more powerful computers. We're using huge
09:25computer farms, for example, to validate the functional correctness of chips or to optimize the physical
09:31implementation. InternalGoal955 is asking, AI conquered software coding and hardware design is next. How do we
09:41prepare for inevitable displacement? Well, I don't really look at it that way. I think the word conquered is
09:47really strong here, too strong. AI tools are really powerful tools that make us engineers more productive.
09:53That's true in software engineering. That's also true in chip development and hardware engineering. But
09:58it's another set of powerful tools that we're building here that makes us better and allows us to build
10:04better chips going forward. I don't think it's going to displace us. It's going to make us more productive
10:09and enable us to build better chips. Exxon North asks, why is silicon so important in the manufacturing of
10:16computer chips? Is there any viable alternative? If not, why? So modern manufacturing processes
10:23for semiconductors, in particular computer chips, cell phone chips, etc., are based on silicon. That is
10:30the technology that has evolved very far in terms of how many transistors we can put on a chip, how
10:36power efficient these chips can be, how we can manufacture them in a very reliable way. There are
10:41different elements in the periodic system that can be used as semiconductors. Silicon is one of them,
10:47germanium is another one. But for the most powerful computer chips, we're really dependent on silicon. A
10:52semiconductor is a material that doesn't conduct electricity like metal does, but that can be
10:59configured to sometimes conduct and sometimes not conduct. That's the word semi. So you can build a
11:05transistor with the gate and depending on what signal you put to the gate, the semiconductor is
11:10either conducting or not conducting. That is the fundamental building block for modern chips.
11:16Rodabi asks, what advancements are made every year that allow us to make faster processors? A whole slew
11:23of things across the whole stack of chip development. The silicon node that's at the base, like is it a
11:285
11:28nanometer, a 4, 3, 2 nanometer chip, that changes all the time. Then microarchitects like myself were
11:35inventing new ways to connect all these transistors and build faster processors at the microarchitecture level.
11:41They're figuring out how to make memory faster, how to make storage faster, how to make network faster.
11:46And then the combination of all those things, computers are getting faster, faster and faster.
11:50A microarchitect is one discipline in the broad field of computer engineering. A microarchitect is
11:55somebody who basically lays out the big picture architecture of the chip before it then gets built
12:02into the different components and subunits that end up making up the billions of transistors.
12:07Dude with the Bling asks, theoretically, how small can a microchip be fabricated?
12:13If you go back to computers from the 1930s and 40s, they were built using magnetic relays or vacuum tubes.
12:21Then in the 50s and 60s, we developed integrated circuits with the transistors on silicon chips,
12:26for example. In the span of my career over the last 25 or so years, we've moved from over 100
12:33nanometer transistors to five and two nanometer transistors nowadays. There's no really strict
12:38limit on how far we can continue to drive this, but there's research going on here, right? Nobody knows
12:43exactly how we'll build these chips in 10 years or 15 years because there's going to be some scientific
12:49breakthroughs. But let me tell you, 15 years ago, people didn't know how we would manufacture the chips
12:55that we have today with two nanometers. That was an unknown. So I believe we'll see the innovation continue
13:01and research breakthroughs enable us to continue to shrink the transistors and therefore add more
13:06and more transistors on each of those chips. So we are now at two nanometers and we're entering
13:12really the research for the sub one nanometer time frame. We're calling that the angstrom age
13:18and we're really now talking about transistors of the size of just a few atoms.
13:24R2002 asks, semiconductor super cycle, are we peaking or just starting? Crash coming?
13:30Well, who knows? As we've talked about, we're building massive new data centers and that is
13:35driving a lot of demand. And it's really hard to build additional supply for chip manufacturing just
13:42because these fabs are so enormously complex and expensive. So what we're seeing is a demand search
13:47from the new data centers and a bit of a supply crunch because it's hard to build more manufacturing
13:54fabs. How that plays out over the next few years is anybody's guess. Microchips have always gone in sight.
14:00We're in a super cycle. Memory cost, for example, has always gone up for a few years, gone down,
14:05gone up again. Right now, we're in what we call a super cycle. With all the construction of new data
14:10centers, there's so much demand for microchips, memory processors, GPUs, and it's really hard to scale up
14:17the manufacturing capabilities because these fabs are so incredibly expensive that we're really seeing
14:22a surge in demand driving the current cost of the microchips up. Are we peaking? Are we crashing?
14:30That's really anybody's guess. I personally believe AI is such a transformative technology that this
14:36cycle is going to continue for a while. Doomkris asks, if you can't put any more transistors on
14:42a microchip because the transistors are physically too small, why don't we just make bigger microchips?
14:47There's physical limits to how big we can make chips, but then also there's commercial limits.
14:52The bigger the chip, of course, the more expensive it is. But let's talk about the physical limits.
14:56When manufacturing chips, we're using masks to create the fine structures on the silicon wafer.
15:03And these masks can only be produced in a certain size. And so building chips above 750 or 780 square
15:11millimeters, it's really hard. And those chips are already very large and therefore expensive.
15:18AICaday is asking, what is the difference between a GPU and CPU?
15:23So let's step back. There's many different types of chips. There's memory chips, there's chips in a
15:30camera that recognize the light and turn the light into electrical signals, etc. A CPU is a historically
15:37very versatile type of microchip that is programmable and can execute all kinds of software. That's really
15:43the heart of your laptop, for example, or the heart of a traditional server computer.
15:49GPUs are a different specialized kind of chip. They came about maybe 20 or so years ago and really were
15:56designed for graphics used, for example, in either gaming or in applications like computer-aided design.
16:03It turns out that the capabilities that you have in GPUs, foremost like real, strong,
16:09high-performance computing capabilities are also very relevant to AI processing. And so the modern AI
16:16models have actually been kind of built around the GPUs because the math at a certain level is similar
16:23to the kinds of math that you do in graphics processing.
16:27Programmer7 asks, could someone explain all the different types of chip design engineers and the differences?
16:32Well, I don't think I can explain all the different types, but I can give you a good taste of
16:36it. It
16:37starts with the people who develop the silicon process, like how the silicon node and the chip
16:42manufacturing works. And then we have the engineers who design the chips. It starts with a microarchitect who
16:48sort of lays out the big picture of how the chip should work. Then logic design engineers implement the
16:54different functions in the chip, the floating-point units and the caches, for example. Verification engineers
17:00make sure that the logic design is functionally correct and produces the correct results when
17:05it computes on the data. Physical design engineers take the logic design and turn it into what we call
17:11a layout. It's really like where do which transistors go? Which function goes where? How is it all interconnected
17:17using the metal stack? And then as the chip gets manufactured, you have all sorts of engineers
17:22and disciplines to actually put a system around the chip. So take this AI accelerator card. Somebody designs the
17:29the module on which the chip sits. Somebody puts it all together and validates it. And you have design for
17:35test engineers who make sure that the chip and the card works from manufacturing. So you have all these
17:41disciplines that bring it all together and make sure that we have functioning computers in the end.
17:47Hyros IT asks, what were the tech leaps that make computers now so much faster than the ones in the
17:531990s?
17:53Really, computer engineering has so many facets and everything gets better all the time. So it's
17:59faster transistors, smaller silicon nodes. It's better design in the processors themselves. It's faster
18:05memory, faster network, faster storage. Everything gets better. If you kept one thing the same as it was
18:11in the 90s, your computers today would still run very slow. So it really takes all of it to come
18:17together
18:17in a full system design to create these breakthroughs. HoodinComplete3081 asks, why does Moore's law keep
18:25ending every decade while computing power somehow keeps exploding anyway? Moore's law was postulated,
18:33not really as a law, but more as an observation that about every two years, we can double the number
18:38of transistors that we can put on a chip. That law is still around. I mean, it still kind of
18:44works,
18:45despite it has slowed down a little bit, right? We're not doubling every two years,
18:49but we can continue to grow the numbers of transistors you can put on a chip. What really
18:54has broken down is Denard scaling. Denard scaling was a rule that you can make transistors smaller
19:00and smaller, put more of them on the chip, and because of the transistors getting smaller,
19:04they end up consuming the same amount of power than the less transistors in the prior generation.
19:10That scaling has really ended. And as we're putting more and more transistors on,
19:15it's really hard to stay in the power budget for the chips that we're designing. And so that's why
19:20you're seeing, for example, processors consuming more power now than they did 10, 15 years ago. So
19:26with chip design now, one of the most challenging aspects is how do we manage the power consumption of
19:31the chip. With the Denard scaling no longer working, as we put more and more transistors into a chip,
19:36they consume more and more power. And so there's a few key challenges here. First, to get the power
19:41into the chip, and then that power creates heat. And so we need to extract the heat. And that's why
19:46you
19:47see fans in your computers. But that's also when you look at big data centers, you see massive power
19:53lines go into the data centers. And then you see cooling towers, for example, they use a lot of water
19:58to
19:58cool the air in the data center, or to even bring cold water directly to the chips to cool the
20:04chips with water.
20:06Bones4Y is asking, how are microchips made with no imperfections? I'll tell you the truth,
20:11when you're designing a chip with billions of transistors, there will be imperfections. And
20:16we're designing to deal with the imperfections into the chip design. So for example, when you're
20:22designing a memory element, you are not just designing the say, one megabyte of memory, you're designing maybe
20:2910% more, then you have switches inside where you can block out a bad memory cell,
20:34and use a spare cell that we have put into the chip. Or think of some strange numbers of cores
20:41on a chip, like you could have a chip with 28 cores, for example. Well, typically, what you would
20:47find is there's actually 30 cores on the chip. And then we look at which of these cores are actually
20:53working. And if only 28 of them are working, we can sell that as a 28 core chip. If only
20:5916 are working,
20:59you could sell it as a 16 core chip. So we just need to prepare for that, have redundancy built
21:04in,
21:05and then structure the offerings so that we can also sell partial good chips.
21:10United Nobody 2532 things, putting chips in people's brains would be great.
21:16Well, let's separate what's actually happening today versus what might happen in the future versus some
21:22science fiction. We've put chips into the human body for decades already. Think of a pacemaker. The pacemaker is
21:28a microchip. It measures the electric signals in your heart and it recognizes something that is
21:33not working right. And it can send a pulse to make the heart beat. Modern pacemakers also contain
21:41memory and take traces and are sort of like a ECG inside your body that can be read out at
21:46a doctor's
21:47office. We have these kinds of things. We have hearing aids. There's already research happening,
21:51for example, to have artificial eyesight where a camera is connected. The microchips and the camera
21:56can be connected into the visual cortex of the brain. So we're seeing a lot of these things where,
22:02I'll just say loosely, we can mitigate disabilities or we could have situations where like, you know,
22:08a patient has a stroke and a chip could be used to repair certain sections of a damaged brain,
22:14for example. That already is happening and a lot of research is happening in that space as well. Where it
22:19gets a bit more complex and controversial is when it comes to actually enhancing the capabilities of
22:26the brain. To me, the brain is a finely tuned instrument that has emotion and intuition and
22:32experience and knowledge. And it makes us think, it makes us be innovative and makes us human. And
22:38I don't know whether, you know, putting an additional chip that could overload the brain with all the
22:43information that's out on the internet would actually help or hurt. It might just overload the brain,
22:47besides all the ethical concerns it would create. A Reddit user asks, why does making chips require
22:54clean facility? Modern transistors are nanometers in size. A dust spec is thousand times that.
23:02Imagine as you're producing the chip that you have a dust spec settle on the wafer and blocks out
23:09thousands of transistors. Well, then the chip won't be able to work. That's why chip manufacturing
23:14facilities are a super, super clean room so that you don't get the contaminations onto the chips that
23:20you're producing. DataNurse47 asks, those who develop chips, what was your career path like? Well,
23:27like in any industry, there can be many different career paths. Mine started as a computer science
23:32student at Saarland University in Germany. And then I joined the IBM Development Lab in Böblingen in
23:38Germany. And I kind of learned chip design as part of my job. I then had an opportunity to move
23:44to New
23:45York and develop next generation mainframe chips. And from there, I kind of grew and went through
23:51different aspects of different chips. I designed processor cores, I designed caches, I designed IO
23:56circuits. I kind of moved around all sorts of different areas. And then as my responsibility and
24:03frankly, my experience grew, I ended up in my current role as CTO. So I started as a computer
24:08scientist. Many engineers start with an electrical engineering background. As a computer scientist,
24:13I was more thinking in terms of how programming works. And then I learned the electrical engineering
24:19engineering part as part of doing my job. So those are all the questions for today. Thanks for watching.
Comments