Artificial Intelligence and Data Center Correlation

Artificial intelligence has quickly become part of everyday work. We all know that. From chip designers to AI researchers, the focus is on turning AI into a practical assistant that helps people move faster and do more. This growing convenience, however, often hides what is happening behind the scenes at the infrastructure level.

A study by the Environmental and Energy Study Institute (EESI) reports that a medium-sized data center can consume up to 110 million gallons of water per year, primarily for cooling. Today, we have the privilege of assigning an AI assistant to our daily work for around 20 bucks month, gaining a noticeable boost in productivity. That privilege is supported by large physical systems that consume significant amounts of water and energy. To maintain availability close to 99.99999%, cloud providers like AWS or Azure operate hundreds of data centers worldwide, increasing the overall physical resource footprint behind what appears to be a simple digital service.

What Happened Before the AI Boom

These water-usage numbers for a mid-sized data center are recent (June 25, 2025), which raises a natural question: what was different before the AI era? Power and cooling have always been part of data center infrastructure, so that alone is not New. The difference lies in how these systems are used today.

The underlying physics has not changed. GPUs, servers, racks, and network switches still obey the same physical laws. What has changed is usage intensity. The AI boom has pushed infrastructure utilization much higher and for much longer duration. Every deep learning or generative AI workload ultimately runs on physical bare metal, regardless of how much virtualization or abstraction is applied on top.

This shift—from mixed, bursty workloads to sustained, compute-dense workloads—means systems now operate closer to their thermal and power limits for extended periods. As a result, physical constraints that were once manageable have become dominant factors in system performance.

Short answer: thermodynamic complexity, details discussion here.

Why Simulation Became Necessary

As computing intensity increases, heat generation becomes unavoidable. When heat is produced faster than it can be removed, temperatures rise beyond safe limits. This leads to performance degradation and potential IT hardware damage. In computer systems, this problem is well known and often appears as localized high-temperature regions, commonly called hot zones.

Because of this, reducing hot zones has long been a concern in data center design. Research work by Long Phan and collaborators, published as early as 2017—well before the current AI-driven scale-up—anticipated challenges that now shape modern data center and colocation operations. Their studies examined airflow and cooling behavior using hot- and cold-aisle layouts, which are common in data centers.

To study these effects, the researchers used physics-based simulation tools such as EnergyPlus, developed by the National Renewable Energy Laboratory (NREL). EnergyPlus was used to evaluate temperature variation and cooling performance under different operating conditions.

This work matters because there are infinitely many possible ways to design, locate, and operate a data center. Small differences in layout, airflow strategy, materials, or operating settings can lead to very different long-term outcomes. Today, these simulation-based methods directly support planning and operational decisions, as AI workloads push physical infrastructure closer to its limits.

Why Simulation Matters for Return on Investment

In data centers, return on investment (ROI) is mostly decided before the first RACK is installed. Simulation helps at this early stage, when design changes are still cheap and flexible. Instead of guessing, teams can test how layout, Heating, Ventilation, and Air Conditioning (HVAC) strategy, and operating conditions affect energy use and long-term cost.

Simulation also makes future costs visible. By modeling energy consumption, HVAC behavior, cooling demand, and utility expenses over time, planners can see how today’s decisions shape tomorrow’s operating cost. This reduces trial-and-error, avoids over-design, supports better capital planning and maximize ROI.

Digital Twins build on this idea. They are a more integrated form of simulation—a virtual representation of a physical system. Using a digital twin, planners can evaluate designs from GPU power management to RACK layout, network switches, and power infrastructure before capital is committed. Explore NVIDIA AIR—you might find it interesting. This supports investment planning by reducing uncertainty around cost and performance.

Physics-based tools such as EnergyPlus support this process by modeling heat transfer, airflow, and system behavior using physical principles. Used this way, simulation becomes a practical planning tool rather than an academic exercise—helping teams make informed technical and financial decisions early.

Physics Still Sets the Limits

Much of the work on hot zones, peak load, data center thermal behavior is grounded in physics-based simulation, not abstract models. Tools such as EnergyPlus matter because they explicitly model heat transfer, airflow, material properties, and boundary conditions using physical principles. This makes them suitable for studying how hot zones form, persist, and shift across different parts of a data center as workloads change.

As AI workloads run hotter and longer, the usefulness of physics-based models becomes easier to see. Work that once focused on isolated thermal cases now helps answer real, day-to-day questions about layout choices, cooling strategies, and long-term efficiency. In this way, physical simulation tools link earlier research to today’s planning and operational needs—not by changing the physics, but by applying it at the right scale.

Hardware–Software Co-Design and Facility-Level Constraints in AI-Centric Data Centers

Historically, software systems were developed with limited awareness of the hardware on which they executed. Abstractions such as virtual CPUs, virtual memory, operating system schedulers, and cloud-based resource allocation mechanisms allowed developers to focus primarily on application logic rather than processor layout, memory hierarchy, or data movement. For many conventional workloads, this abstraction model was sufficient.

Artificial intelligence changes this assumption.

Modern AI workloads place extreme pressure on compute, memory, networking, and power infrastructure. Training and inference tasks require massive parallelism, high memory bandwidth, low-latency data movement, and efficient utilization of specialized accelerators. As a result, the separation between software design and hardware architecture is becoming increasingly impractical. In AI systems, performance is no longer achieved by software optimization alone; it depends on the alignment between software execution patterns and the physical capabilities of the hardware.

AI workloads are highly parallel by nature. Matrix multiplications, tensor operations, and large-scale model execution benefit from thousands of concurrent execution units. However, software cannot create parallelism beyond what the hardware can physically support. If the underlying system lacks sufficient cores, memory bandwidth, cache efficiency, or interconnect capacity, software-level optimization provides only limited improvement.

Modern GPU architectures demonstrate this principle clearly. For example, AI accelerators depend heavily on high-bandwidth memory placed close to compute units. This proximity reduces data movement latency and helps keep a large number of processing cores active. When data spills from accelerator memory into slower system memory, performance can degrade significantly even when compute resources remain available.

Architectures such as NVIDIA Hopper reflect this shift toward tighter integration between compute, memory, interconnects, and power efficiency. In these systems, performance is not determined by raw compute capability alone. It is determined by how effectively the software workload maps onto the hardware’s memory hierarchy, data paths, and parallel execution model.

Therefore, hardware–software co-design is no longer an optional optimization strategy for AI. It is a core requirement for achieving practical performance, energy efficiency, and scalability.

Power and Physical Constraints in AI Data Centers

AI workloads do not only stress computing hardware. They also stress the physical infrastructure of data centers.

Many traditional data centers were designed around assumptions such as moderate rack density, predictable thermal distribution, and intermittent peak utilization. AI workloads violate these assumptions. GPU-dense servers generate sustained and concentrated heat, while also requiring significantly higher power delivery per rack than conventional enterprise infrastructure.

A traditional redundancy model, such as a basic N+1 design, may appear sufficient on paper. However, under sustained AI workloads, facility-level weaknesses can become visible. High-density GPU systems may create localized thermal pressure similar to a physical noisy-neighbor effect. Hot spots can form, airflow can become uneven, and nearby equipment may experience thermal stress. Over time, these conditions can reduce performance, increase failure rates, and shorten hardware lifespan.

These problems cannot be solved by software alone.

Orchestration platforms such as Kubernetes can reschedule workloads, restart containers, and distribute tasks across available nodes. However, they still depend on functioning physical servers. If a server degrades or fails due to thermal stress, the orchestration layer loses the underlying resource. Once the physical infrastructure fails, the software stack cannot compensate.

For AI workloads, resilience begins at the facility level. Power delivery, cooling capacity, airflow design, rack density, and thermal monitoring must be treated as first-class design concerns.

Economic and Operational Barriers to AI Data Center Ownership

The infrastructure requirements of AI explain why most startups, and even many large organizations, do not operate private AI data centers.

AI-capable facilities require large capital investment, high power availability, advanced cooling systems, specialized operational expertise, and long planning cycles. In some cases, the power demand of an AI data center may be comparable to that of a small town. Beyond technical challenges, organizations must also deal with permitting, regulatory approval, grid coordination, environmental review, and compliance requirements. These processes can take longer than the product development cycle itself.

As a result, many organizations rely on public cloud providers such as AWS, Microsoft Azure, or Google Cloud. Others use co-location facilities, where they rent rack space, power, cooling, and network connectivity while deploying their own hardware. In both models, physical constraints are defined contractually and must be evaluated before deployment. For AI workloads, power availability, rack density, cooling capability, and expansion capacity directly influence cost, scalability, and long-term feasibility.

Cost remains the dominant factor in these decisions. Cloud users generally select infrastructure based on the ability to run workloads at the lowest acceptable cost. If the same AI workload, such as a biological simulation or de novo modeling task, costs less on one platform while offering similar reliability and availability, the lower-cost option is usually preferred.

This cost pressure affects infrastructure providers directly. AI servers must remain highly utilized to generate revenue. Even when idle, they still consume power, cooling, space, and maintenance resources. If utilization decreases, profit margins can decline quickly. Therefore, efficiency in AI infrastructure is both a technical and economic requirement.

Choosing Between Traditional and AI-Centric Data Centers

A central question for infrastructure planning is whether an organization should continue using a traditional data center model or move toward an AI-centric facility design. The answer depends primarily on organizational role and workload characteristics.

The organization’s role matters. For application developers, facility-level design may not affect daily development work directly. Their main concerns are usually software performance, deployment reliability, and cost. However, for cloud providers, private data center owners, and co-location operators, infrastructure design directly affects utilization, pricing, reliability, and return on investment. In these cases, data center architecture is not only a technical decision; it is also a business decision.

Workload characteristics matter even more. Facilities running GPU-heavy AI workloads must support sustained high power draw, concentrated heat output, and dense accelerator deployments. Traditional air-cooled designs may be insufficient for these environments, especially when workloads run continuously at high utilization. By contrast, facilities running conventional enterprise workloads may not require AI-centric cooling and power infrastructure. For those workloads, traditional data center designs may remain cost-effective and operationally sufficient.

Therefore, the decision should not be based on whether AI-centric infrastructure is generally better. It should be based on workload intensity, power and cooling requirements, expected utilization, expansion needs, and cost structure. For AI-heavy environments, infrastructure must be designed holistically across software, hardware, and facilities. For conventional workloads, a traditional data center model may still provide the best balance between cost and performance.

Final Remark

AI workloads are reshaping the design principles of modern computing infrastructure. The traditional separation between software, hardware, and facilities is becoming less effective as performance increasingly depends on their alignment. Hardware–software co-design is essential for extracting performance from modern AI accelerators, while facility-level design is essential for sustaining that performance reliably.

At the same time, infrastructure decisions are governed not only by technical requirements but also by economic constraints. Power, cooling, utilization, and pricing determine whether AI infrastructure can operate profitably at scale.

Therefore, the decision to adopt a traditional or AI-centric data center model should be based on workload intensity, power and cooling requirements, business role, and cost structure. For AI-heavy environments, infrastructure must be designed holistically across software, hardware, and facilities. In modern AI systems, performance and resilience are no longer purely computational problems; they are full-stack infrastructure problems.

In the next article, we will focus on why forecasting matters more than observability for AI infrastructure, and how early modeling changes the economics of scale before deployment, not after.