GTC 2026 highlights hyperscale and mid-market AI infrastructure

As impressive as the GTC keynote was, it only captured a small portion of what we’re seeing.

The larger picture

The AI factory narrative NVIDIA presented at GTC is accurate for hyperscalers. It reflects how the largest cloud providers and technology companies are thinking about infrastructure at extreme scale. However, it does not describe the majority of organizations building and deploying AI infrastructure systems today.

87% of PNY’s customers – PNY being one of NVIDIA’s primary distributors – run fewer than ten DGX systems. The most impactful medical AI programs in the UK are running on six DGX systems. Conservation AI at a global scale is running on two GPU servers. 

This isn’t the fringe of the market. It’s the mainstream.

This pattern is consistent across previous infrastructure waves. The headline numbers tend to describe the top end, where scale and capital expenditure are highest. The broader market typically develops in the middle – organizations with serious requirements and budgets, but no appetite for hyperscale complexity. That’s where a significant portion of long-term adoption takes place.

Storage: Identified, but not fully addressed

One of the more notable aspects of this year’s keynote was Jensen explicitly naming storage as one of the five pillars of the AI factory, alongside compute, memory, networking, and security. That framing reflects a growing recognition of storage as a first-order concern in AI system design.

However, the discussion largely stopped at identification. The practical question – what purpose-built AI storage looks like for organizations operating outside hyperscaler environments – didn’t come up in the keynote, despite being a key topic in every infrastructure conversation.

In many deployments, GPU utilization falls short of hardware capacity, not because the GPUs are wrong, but because the storage systems feeding them were not designed for AI workload profiles. For organizations running 10, 15, or 20 GPUs, this can become a persistent bottleneck. It is rarely visible on a specification sheet but shows up every day in performance that falls short of what was promised.

These challenges are not new, and in many cases, they are or have already been solved. The issue is less about the existence of solutions and more about their adoption across the broader market.

Ongoing memory constraints

Another significant statement from GTC came from the sidelines, rather than the keynote stage itself. SK Group Chairman Chey Tae-won, whose company SK Hynix is NVIDIA’s primary HBM supplier, said that the industry-wide memory supply shortfall will persist at over 20% through 2030, meaning four to five years of elevated prices and constrained supply.

For many organizations, this changes the infrastructure equation entirely. When hardware refresh cycles become significantly more expensive and supply is constrained, the imperative shifts toward extracting more performance and efficiency from existing infrastructure. In this environment, software-defined storage that delivers AI-grade performance from commodity hardware isn’t a workaround. It’s the right architectural answer. 

What this means for the broader market

The GTC coverage cycle is dominated by Vera Rubin benchmarks, hyperscaler deployment announcements, and the $1 trillion order book. These are important indicators of where the industry is heading at the highest level, and it’s understandable that much of it is what the media is focusing on.

However, the story that matters more to the majority of enterprise IT leaders, research institutions, and domain-specific AI teams is the one GTC quietly confirmed through its session catalogue and show floor: AI infrastructure at a smaller scale is maturing rapidly. DGX Spark was on sale at the show; NemoClaw runs on a laptop.

The capability is moving down the stack. Systems are becoming more accessible, more modular, and easier to deploy outside of hyperscale environments. Edge and near-edge use cases are clear examples, as constraints on power, space, and latency require a different approach to infrastructure design.

The reality is that the AI infrastructure market is not defined by the largest deployments – a point GTC 2026 both highlighted and, at times, overlooked. While Jensen Huang’s keynote focused on hyperscale systems, GTC as a whole reflected a much wider range of real-world adoption. 

This being said, the most important developments in most of the market might not be the largest systems described on stage. Instead, they’re the continued progress in making AI infrastructure usable, efficient, and effective across a broader range of real-world environments.

About the author

Roger Cummings is the CEO of PEAK:AIO, a company at the forefront of enabling enterprise organizations to scale, govern, and secure their AI and HPC applications. Under Roger’s leadership, PEAK:AIO has increased its traction and market presence in delivering cutting-edge software-defined data solutions that transform commodity hardware into high-performance storage systems for AI and HPC workloads.

Article Topics

AI infrastructure  |  GPUs  |  semiconductors