From the core to the edge: scaling networks for AI’s future
By Mattias Fridström is the Vice President and Chief Evangelist at Arelion
Many conversations in the telecommunications industry currently focus on the hyperscale and cloud data centers operating AI training workloads, with the network core playing an integral role in facilitating high-capacity data transfer between these data centers. Eventually, these workloads will shift to the network edge to enable AI inferencing. This function will reshape business functions across diverse industries, allowing companies to utilize pre-trained AI models to process requests at edge sites closer to end users. Although inferencing is less bandwidth-intensive than AI training workloads, it will still drive Internet carriers to optimize their long-haul infrastructure and networking sites by reducing latency and enhancing scalable capacity, helping them support this emerging use case.
Market forces and infrastructure enhancements
Analysts project that accelerated servers optimized for AI will comprise nearly half of the data center market’s $1 trillion CAPEX by 2029. In turn, Internet carriers’ architectural transformations must support several crucial networking qualities so enterprises and hyperscalers can maximize their AI investments. However, these dynamic, latency-sensitive workloads pose bottleneck risks and other challenges to traditional networks. As data centers increase their investments in accelerated GPU and TPU servers, their infrastructure generates and consumes massive data sets, putting additional pressure on network links. So, how will inferencing likely transform network infrastructure to reduce latency, jitter and other risks?
Inferencing has similar requirements to Content Delivery Networks (CDNs), including the need for fast, localized delivery. However, AI inferencing is more dynamic and less cacheable due to its context-specific nature, making reliable network performance more critical to its real-time operations. Let’s explore how telecom operators can meet AI inferencing’s decentralized demands by optimizing key networking qualities, including reach, capacity, scalability and more.
A reliable and expansive footprint
As with CDNs, backbone networks will prove critical in distributing inferencing responses to end users through Points-of-Presence (PoPs) that provide optimized connectivity in major and burgeoning markets. Ultimately, inferencing will rely on an expansive reach that allows carriers to localize AI workloads and provide access to over 70,000 networks that comprise the global Internet, ensuring low-latency delivery to end users.
Reliability is another key networking facet in supporting this technological evolution, enabling companies to leverage high-availability services to deliver model outputs to the edge. Internet carriers can improve reliability through network diversity and latency-based segment routing, allowing them to route customers’ AI traffic through the next best, low-latency path in the event of a service disruption. This quality is critical amid rising geopolitical sabotage, weather-related outages and accidental fiber cuts which threaten real-time AI operations.
Maximizing scalable capacity through optical innovation
Amid data center innovations to support emerging applications, Internet carriers are also transforming their optical networking infrastructure to enable AI use cases through scalable capacity. Carriers are increasingly integrating 400G coherent pluggable optics in backbone networks by leveraging open optical line systems, allowing them to meet their customers’ capacity and scalability needs. Unlike legacy architectures that rely on traditional transponders, coherent pluggables offer a modular, software-driven approach that aligns with the distributed, dynamic attributes of AI workloads and their real-time capacity requirements.
While inferencing will occur at the edge, training data must still be sent back to core and cloud networks for aggregation and analysis purposes. 400G coherent pluggables (and 800G pluggables on the horizon) enable core-edge synergy through high-capacity links between core, cloud and edge nodes, allowing carriers to support AI’s fluctuating data needs. Amid AI’s massive energy demands, these pluggables also reduce space and power consumption compared to traditional transponders, helping carriers improve the cost-efficiency and sustainability of their networking infrastructure.
No matter the scenario, backbone connectivity remains crucial
While AI workloads are typically concentrated in hyperscale and cloud data centers for now, inferencing marks the next phase of AI’s evolution. Backbone connectivity’s vital utility for AI data transfer between data centers is well established. Still, companies must remember that backbone connectivity will also prove critical in supporting eventual AI functions at the network edge. By maximizing these key networking qualities, Internet carriers can provide the foundation for AI inferencing, helping hyperscalers, cloud data center operators and enterprises unlock AI’s business value through scalable, reliable connectivity.
About the author
Mattias Fridström is the Vice President and Chief Evangelist for Arelion. Since joining Telia in 1996, he has worked in several senior roles within Telia Carrier (now Arelion), most recently as CTO. He has been Arelion’s Chief Evangelist since July 2016.
Rakuten Symphony and Tejas Networks partner to break telecom lock-in with Open RAN
Article Topics
AI inferencing | AI networking | connectivity | data center | digital infrastructure | edge computing | edge networking | network edge
Comments