2023 AI and Cloud DC Networking Showcase

Nathan Tracy, President of OIF and technologist at TE Connectivity, underscores the growing significance of machine learning and artificial intelligence in data centers, necessitating higher bandwidth and bandwidth density that will impact power consumption and thermal management. To tackle these issues, OIF is developing a 200 Gbps (Gigabit per second) electrical interfaces, a 112 Gbps linear optics specification, and Energy-Efficient Interfaces (EEI), along with working on CMIS, the common management interface specification, and specifications for 400ZR, 800ZR, and 1600ZR, all crucial for next-generation data centers.

Brad Booth, an independent Ethernet and Optical Technology Advisor, discussed the rapid expansion of AI data centers, driven by AI's increasing capabilities and its adoption by hyperscale and cloud market players. He highlighted the high demand for GPUs and optical modules, the impact on memory devices, and the surge in innovation, including the exploration of new materials for optics and memory elements to meet bandwidth demands.

Siamak Tavallaei, CXL Advisor to the Board at Open Compute Project, emphasizes the importance of CXL in enabling interconnection of large systems for machine learning and artificial intelligence, and the need for a fundamental standard for interconnectivity. He asserts that CXL can facilitate the interconnection of 10,000 units, petabytes of storage, and hundreds of terabytes of memory in a shared environment, promoting device-to-device and host-to-host communication.

Mansour Karam, VP of Data Center Products at Juniper, discusses the growing data center market, expected to reach $32 billion in four years, and the significant role of AI/ML workloads in this growth. Karam emphasizes the need for software that can optimize job completion time and automatically tune parameters in AI/ML workloads, a solution that Juniper, with its expertise in AI and data feeds, and Apstra, a company Karam founded and Juniper acquired, can provide.

Sanjay Kumar, VP of Products and Marketing at Arrcus, introduced the company's solutions for enhancing AI and 5G capabilities for network operators, including data center solutions — a network fabric overlay FlexMCN and SRv6 mobile user plane (MUP) technology developed for SoftBank Mobile. Kumar also highlights ArcIQ, a solution offering end-to-end network visibility, and discussed the role of 5G as a transport mechanism for AI and ML workloads, emphasizing the company's commitment to optimizing GPU utilization, minimizing latency, and reducing total cost of ownership.

Kamran Naqvi, Principal Network Architect at Broadcom, discussed the company's efforts to democratize networking by making features previously exclusive to hyperscalers available to all users, including the Sonic software, an open-source network operating system. Broadcom has enhanced Sonic to be more enterprise-class, leading to increased adoption among enterprise customers, and has enabled silicon telemetry features in its silicon, which can help customers quickly identify the root cause of network or application performance issues.

Shawn Hakl, VP of 5G Strategy at Microsoft, emphasized the potential of integrating operations community with software-based solutions in networking, which can enhance customer experiences, improve network management efficiency, and enable the scaling of previously complex services. Hakl believes this approach, which involves managing the network as an API, can modernize and monetize the network in innovative ways.

Charlie Ashton, a Senior Director of Business Development at Napatech, discusses how cloud and enterprise data centers need IPUs (infrastructure processing units) to accelerate networking, storage, and security workloads while freeing up host CPUs for applications and improving security through workload isolation. Napatech provides complete IPU solutions comprising hardware and software that deliver high bandwidth and accelerated performance for infrastructure workloads via dedicated processing, offloading host servers and introducing workload separation for greater security.

Broadcom's GM and VP of Optical Systems Division, Near Margalit, announced that the company has successfully tackled the main technical challenges of deploying CPO switches and is now focusing on demonstrating the reliability and cost-effectiveness of the core silicon photonics technology. Margalit also addressed concerns about the reliability of the laser component and the time to market, stating that Broadcom is working to shorten the time between the availability of CPO technology for integration with the switch and its market launch.

Hani Daou, Business Development Manager at Multilane, emphasized the need for data center upgrades due to increasing compute power requirements, with investments being directed towards technologies like 100 terabit AS6 and 200 gigabit per Lambda Optics. Multilane, currently experiencing high demand for its 800 gig systems and copper interconnect test solutions, is well-positioned to support semiconductor vendors, cloud service providers, and interconnect vendors with a strategic roadmap for R&D investment and bandwidth scaling.

Radha Nagarajan, SVP and CTO of Optical and Cloud at Marvell, emphasized the importance of optical connectivity in managing data transport as AI clusters increase in size. With the introduction of a 1.6 DSP for the 1.60 generation, Marvell is ready to meet the growing demand for optical connectivity solutions in AI-centric data centers.

Sameh Boujelbene, Vice President of Dell'Oro Group, highlights the emergence of large AI applications necessitating a new AI backend network, presenting a significant market opportunity. By 2027, two-thirds of these backend network ports will be 1.6T, and all will be at least 800 gig, indicating the industry's growing interest in the AI Network market, which will be further explored in an upcoming AI Network report.

Sean Varley, VP of Product Marketing at Ampere, underscored the importance of optimizing AI models for power efficiency and the need for large-scale computing in the context of AI evolution. He also unveiled Ampere's strategy to create CPUs capable of efficiently managing inference at reduced cost and power, citing the recent launch of their 192-core Ampere One Parts that can provide high computational power at less than 200 to 400 watts.

Bill Brennan, President and CEO of Credo Semi, forecasts 2023 as the pivotal year for AI, with the interconnect bandwidth bottleneck in the AI sector driving demand for higher bandwidth and denser networking. Credo Semi is concentrating on resolving this connectivity issue for the AI cluster backend network, which has led to a swift global shift to 100 Gig single lane connections and is expected to hasten the development of next-generation technologies.

Alan Weckel, Founder and Technology Analyst at 650 Group, discussed the potential of CXL technology to compete with NVLink, emphasizing its importance in enhancing servers for AI and machine learning. Weckel also noted the rapidly growing demand for bandwidth, necessitating new high switches and multiple switches per rack, and highlighted the shift in spending from cloud to AI, likening it to the significant architectural shift brought about by cloud technology 20 years ago.

Bapi Vinnakota, the ODSA Project Lead at Open Compute Project, emphasized the growing need for more bandwidth and diverse types of compute in data centers, highlighting the efficiency of chiplets in building heterogenous systems. He concluded that chiplet-based systems will be crucial in future data centers, especially those with AI-intensive workloads, and advocated for open interfaces as the optimal way to integrate systems and chiplet devices from various vendors.

David Ratchkov, founder of Thrace Systems and lead of the CDX work stream group under ODSA in the Open Compute Project, has announced the upcoming release of a detailed white paper on chiplet integration workflows. The paper will cover all aspects of chiplet integration and will be available on the Open Compute Project's website once released.

Cliff Grossner, VP Market Intelligence and Innovation at Open Compute Project (OCP), discusses the advancements in the open chiplet economy vision, including standardizations that allow companies to build a chiplet and create an electronic data sheet for others to use in their design tools. Grossner also revealed plans to establish a marketplace for chiplets.

Vapor's CEO Cole Crawford and CMO Matt Trifiro have introduced a new service, Zero Gap AI, designed to simplify the deployment of wireless and AI stacks by offering an end-to-end solution in partnership with Super Micro, leveraging the Grace Hopper super chip and the Nvidia MGX platform.

Yervant Zorian, Fellow & Chief Architect at Synopsys, highlights the potential of cost-effective chiplets over monolithic chips, suggesting the industry is nearing the limits of electronics, thus making a shift to chiplets more likely.