What’s Inside AI Data Centers: Concrete Warehouses Powering the Boom

Asianet News English

Ever wondered what's REALLY inside the data centers fueling the AI revolution? AFP takes you on a rare tour: massive concrete warehouses packed with 19-inch server racks, GPUs hotter than ovens, water-guzzling cooling systems (66B liters in US alone last year!), and power-hungry ops chasing ‘behind-the-meter’ energy like solar, gas, and future nukes. From Ashburn's data hub to rural builds balancing speed vs. cost – we unpack the heat, water wars, and energy crunch as Big Tech races for AI dominance.

Transcript

00:00The expansion of data centers to power the AI boom has more people wondering,

00:07what exactly is in a data center?

00:16Data centers are the physical infrastructure that make our digital lives possible.

00:21Yet most people have never seen one up close or understand how they operate.

00:26Roughly 12,000 data centers are in operation in the world, with about half in the US,

00:32according to CloudScene, a data center directory.

00:35At its most basic, a data center is a concrete warehouse filled with thousands of computer servers working in tandem.

00:43Traditional facilities span one or two floors divided into vast rooms, though newer ones rise higher.

00:50A facility may serve a single company or be shared by several clients.

00:54The servers sit in standardized 19-inch racks, essentially metal closets lined up in rows.

01:01A large data center can house tens of thousands of servers running simultaneously,

01:06generating enormous heat and consuming significant energy for both power and cooling.

01:12High-speed networking equipment, switches, routers and fiber optic cables connects everything,

01:18moving terabytes of data per second.

01:28Having a data center close to end users improves speed, which is critical for things like trading and gaming,

01:35where immediacy is paramount.

01:37Ashburn, Virginia, which has the highest concentration of data centers in the world,

01:43offers ideal conditions as it is located only about 30 miles from the US capital, Washington.

01:50However, building in densely populated areas costs more and faces local resistance.

01:56Companies increasingly turn to rural locations, where land is cheaper and zoning less restrictive.

02:03But distance adds to loading times, that brief delay when a page loads or a feed refreshes.

02:10To balance cost and performance, operators typically house core infrastructure or the training of AI models

02:18in affordable rural regions while keeping equipment that handles time-sensitive requests closer to urban centers.

02:25Inside these bunker-like buildings, a single-server rack generates as much heat as several household ovens running nonstop.

02:40Cooling consumes roughly 40% of a data center's total energy.

02:44The most advanced chips, graphics processing units used for AI, can reach temperatures exceeding 90 degrees Celsius,

02:52threatening performance and causing permanent damage during extended operation.

02:57They are also much heavier than lower-performing chips.

03:00Traditional facilities use computer room air conditioners with heat blasting out of mounted vents on, on rooftops.

03:07But this is not fit for GPUs, that mainly turn to water for cooling.

03:12Modern facilities are beginning to deploy free cooling, that uses outside air when temperatures allow,

03:18and different water-based approaches.

03:20Liquid cooling systems that pump coolant directly to components,

03:25or evaporative cooling that works like perspiration on skin.

03:28Today, massive amounts of water are still required for direct and indirect cooling in data centers.

03:35In 2014, U.S. data centers used 21.2 billion liters of water,

03:41and that number rose to 66 billion liters in 2023, according to federal estimates.

03:48Power supply, and the high-voltage transmission lines needed to source it, is key for a data center,

04:01and is only growing with facilities that run the powerful GPUs.

04:05The big tech giants, caught up in the AI arms race, have spent tens of billions of dollars in just months

04:12towards building suitable structures for GPUs.

04:16Operators rely on the existing power grid, but are increasingly seeking to secure their own resources,

04:22called behind the meter, for greater security, and to limit rate increases for all users.

04:29Solar panels or gas turbines are sometimes installed,

04:32and many are also awaiting the arrival of the first small modular reactors,

04:37a nuclear energy technology currently under development.

04:41Most data centers have to run 24 by 7, and every critical system has backups in case of power outages.

04:48This can come through massive battery banks or diesel generators.

04:52The best facilities guarantee power 99.995% of the time.

04:58of the time.

04:59of the time.

05:00of the time.

05:01of the time.

05:05of the time.

05:06of the time.

05:07of the time.

05:08of the time.

05:09Not only, shockingly.

05:10of the time.

Category

Transcript

Be the first to comment

Recommended