00:00Thank you. Senator Hickenlooper. I appreciate that line of questioning. I was ready for you
00:05to continue as well. I could have given you a minute or two. Mr. Smith, Microsoft has a long
00:11and deep history in transforming workplaces all over the world through software, from Windows
00:17operating system to its office applications like PowerPoint, Excel, and now the AI-powered
00:25co-pilot application. In software development, life cycles seem to be becoming increasingly
00:31shorter, updates becoming more frequent. What are the internal processes that Microsoft follows
00:38to evaluate co-pilot's accuracy and performance before it was released, and what kind of independent
00:44review teams, other than Microsoft's own product developers, are involved in that? Who do you
00:50bring in to help with that? Well, first of all, since most of what we're talking about here,
00:55when you're talking about our co-pilots, start with models that are developed at OpenAI, I would say
01:02OpenAI has its internal process. There is then a joint, what's called the DSB, a Deployment Safety
01:09Board, where we decide together whether something is safe to deploy, as the name implies. We then,
01:17at the applications level, have our own internal deployment safety board. We have a variety of
01:23engineering tools that we use to assess these features. We test these features. We have red
01:31teams, meaning sort of competing teams that often go to work to sort of attack the features, and then
01:37ultimately, the product is released when those tests are completed and the results are satisfactory.
01:43Good. I like that. Well, let me go over to Mr. Altman. The, obviously, you all have a natural
01:51incentive to ensure that the products are high quality and safe, and, but the field is so competitive,
01:57and, you know, in applied research, and with rigorous testing, these constant improvements really are
02:08fundamental steps to the performance of a model, so risk assessments are that key tool, and I'm a big
02:17believer in evidence-based technical standards. I've been accused of being the only real scientist
02:24who's published peer-reviewed papers in the, in the Senate. So, Mr. Altman, do you believe that under
02:32appropriate circumstances, uh, independent evaluations based on, uh, standards performed by qualified
02:38out, evaluators, uh, and done voluntarily, could help validate the testing that you're perform,
02:45performing internally, and in conjunction with, uh, uh, peer companies? Thank you, Senator, and I think
02:51it's awesome that you are, have published peer-reviewed papers. We'd love to see more of that. Well, wait a second. I was,
02:56on the Maslov's Triangle of Science, I was near the bottom, I was a geologist, so that's, you know, I'm not high up in that.
03:02Geology's great. Um,
03:06yes, I think what you say is very important. It's an important part of our process today. Uh, external testing helps us
03:13find things that we may have missed internally, and as, we're, we're very proud of our safety record on the
03:18whole. Not that we, you know, we've not been perfect, and we're continuing to learn new things, but I think we do have a
03:24process that is leading towards models that the public generally thinks are safe and robust to use. Um,
03:30and we've developed a lot of techniques to be able to continue to deliver that, but external testers and red teamers,
03:36uh, are a critical part of that process, and I think they've helped us find many things in the models to improve.
03:43Mr. Smith, would you add anything to that?
03:46No.
03:47Okay, got it. Uh, someone giving testimony who doesn't have something to add.
03:52It's a moment of scientific reflection. Uh, Dr. Hsu, the Bipartisan Chips and Science Act,
03:58uh, historic effort to try and maintain U.S. leadership and emerging technologies like semiconductors,
04:04but others as well. Um, as the technology arms race continues globally, and you were talking about this,
04:10uh, AMD plays a key role in delivering, um, state-of-the-art designs, the best,
04:17uh, for the new chips that are going to power electronics and the devices that are going to allow
04:23AI to become global. Um, as scientists work around the clock to develop new breakthroughs,
04:30um, and to try and increase and improve in performance, but at the same time shorten R and D
04:38timelines, um, what do you see as the next frontier of chip technology in terms of, of energy efficiency?
04:45Uh, and how, and that's not just based on, on the Chinese competitors, but, uh, how can we work
04:52together to improve, uh, direct-to-chip cooling for high-performance computing?
04:57Well, uh, thank you for the question, Senator. I would say, look, there is a tremendous amount of
05:02innovation that's going on in the semiconductor sector today. Uh, the Chips and Science Act was
05:07certainly helpful in raising the profile of chips in the United States, uh, relative to,
05:12you know, what are we doing to, uh, go faster and build, uh, better and more power-efficient chips?
05:18Uh, frankly, we're using AI, um, extensively through our chip development, um, cycles, and it,
05:24it does allow us to augment what are, you know, typically very long cycles, many years, um, you know,
05:29several years for us to develop chips. We can shorten the time and also, uh, improve the efficiency.
05:34And there are lots and lots of, uh, great new technologies in terms of cooling technologies
05:40that are, uh, super important for us to build the large-scale systems that, uh, we talked about
05:44earlier today. So, thank you for the question.
05:46You bet. All right. I'm out of time. I'll yield back to the chair. Thank you all.
05:49And, and Senator Hickenlooper, I will say, as a Texan whose parents were in the oil and gas business,
05:55I think geologists are awesome.
05:57We have a consensus.
Comments