Navigating Data Center Power Density in the Age of HPC and AI

High-performance computing (HPC) is quickly evolving. What used to be reserved for highly technical companies is now being more widely used by the average consumer, thanks to artificial intelligence (AI) applications and large language models. This sudden burst in popularity is also significantly impacting the strain on data centers, making high-density facilities more important. We’ll cover the basics of data center power density, and what to expect as HPC demand continues to grow.

What is Data Center Power Density?

Data center power density, simply put, expresses how much power, in kilowatts (kW), is consumed by a server rack. The greater the density, the higher the number of kW can be accommodated within a single rack.

Why is Power Density Crucial in a Data Center?

When it comes to data center power density, optimization is the core benefit. By being able to facilitate higher-density racks, data centers can reduce their capital expenditures (CapEx) by needing fewer racks, in-cabinet power distribution units (PDUs), cabling, and other equipment to accomplish the same tasks.

The smaller footprint that comes with higher data center density also improves operational efficiencies. Staff will have fewer square feet to manage. Plus, increased densities also mean more opportunities to scale without having to move to a new physical space.

Power density also comes with performance benefits. Shorter cable runs mean lower latency for applications, improving user experience. As IT workloads get more demanding, every piece that improves performance adds up.

How to Calculate Data Center Power Density

Power density can be calculated by taking the total kilowatts (kW) of power that all devices in each cabinet consumed. Multiplying this number by all racks in the facility can help you understand how to size and plan for the future. This number also determines whether a data center is standard density or high-density.

Racks come in various densities, ranging from standard to ultra high-density, each with distinct averages for power consumption:

• Standard density racks can consume up to 10kW of power per cabinet
• High-density racks have cabinets that can consume 10kW to 30kW of power
• Ultra high-density racks can consume anywhere from 30kW to 85kW of power per cabinet

While the 2023 Uptime Institute Global Data Center Survey Report found that average rack densities are less than 6kW per rack, this is expected to grow in conjunction with the increased use of AI, ML, and HPC solutions.

The Rise and Impact of High Power Density on Data Centers

As data centers evolve to meet the demands of modern computing, the integration of high-density racks impacts several areas.

Operational Efficiencies

Data centers that are up to the task of accommodating high-density racks will be able to benefit from operational efficiencies. The reduced footprint of a high-density data center can streamline the networking, wiring, and cross-connect functions, and also bring expenses down.

Costs

Improved operational efficiency in large deployments, especially with high-density data centers, results in direct reduction in cabling materials and installation. Precise environmental controls not only conserve capital but also enhance power efficiencies.

Cooling

Cooling is one of the biggest challenges that high-density data centers have to contend with. Using systems like liquid cooling or liquid-air hybrid cooling technology can lower operating temperatures and keep system performance high. To lower costs, data centers should monitor power and thermal metrics at the cabinet level and deliver resources based on demand.

Types of High-Performance Computing Driving Data Center High Power Density

Despite growth in data center power density being slow in recent years, the rapid and widespread use of compute-intensive workloads and applications by everyday users will, undoubtedly, drive higher rack densities. AI, ML, internet of things (IoT) devices, and big data analytics applications are some of the major contributors to higher-density demands.

Artificial Intelligence and Machine Learning

AI is becoming more commonplace and the introduction of AI in data centers has had a significant impact. Facilities can use AI and ML to automate previously manual tasks, improving productivity and shifting focus to more complex projects. While there is still some distrust among data center operators regarding the abilities of AI to handle operational duties, Uptime Institute has found that most operators believe acceptance will grow.

In the mainstream, AI and ML applications are being used to detect fraud, run image recognition, run large language models (LLM), generate composite visuals, and more. These models pull from a large repository, requiring more memory and processing power for training and deployment.

Internet of Things

IoT devices rely on real-time feedback to operate properly. In the home, IoT devices can include smart thermostats, lightbulbs, and locks. Self-driving vehicles, smart supply chain management, and smart buildings, or even cities, can also take advantage of this same technology.

As the scale of IoT increases, HPC will be more necessary to make decisions quickly and extract insights effectively.

Big Data

What is “big” in big data continues to expand. It’s estimated that by 2025, we will globally create 180 zettabytes of data, tripled from 64.2 zettabytes in 2020. To put that in perspective, one zettabyte is 1 trillion gigabytes, or 1 billion terabytes.

Processing and analyzing large datasets, as more data is being collected to pull from, will require more processing power and memory best served by high-density data centers.

How High-Density Colocation Supports High-Performance Computing

Businesses looking to get the most out of high-density data centers without running their own facility can look to high-density colocation services. High-density managed colocation is built to support HPC and AI workloads with high-density colocation cabinets. Instead of starting from scratch, high-density colocation providers start with increased demand in mind and offer robust power infrastructures with modern and efficient cooling, all in a secure and scalable space.

Choosing the Right High-Density Colocation Provider

Choosing the right high-density colocation provider for your business means that you must consider several different variables:

• Is the location right for your business? Is the facility close to your headquarters or your users? Is it in a geographically diverse area or a place less prone to extreme weather conditions?
• Is the infrastructure secure? You may want to ask questions about on-site personnel, authentication systems, and access control.
• What’s the data center’s redundancy level? The redundancy level of the data center should coincide with the level of coverage your business requires. 
• Is there sufficient resiliency? Are the facility’s power and cooling systems resilient?
• Is there proper monitoring in place? Ask about the prospective provider’s environmental monitoring and infrastructure monitoring measures.
• Is the data center ready to scale with what comes next? You want to think not only about how facilities can meet the needs of today’s HPC, but also their ability to keep up with a greater proliferation of these technologies. Data centers will also need to meet sustainability requirements as they continue to take shape.

TierPoint’s high-density colocation services are ready and able to accommodate the needs of current high-power computing. With ultra high-density colocation options, we’re also primed for exponential growth. Learn more about our capabilities.

FAQs