South Korean AI chip startup FuriosaAI collaborates with Broadcom Inc. to develop the next generation inference accelerator, focusing on memory access and high-bandwidth data transfer.

South Korean artificial intelligence (AI) chip startup FuriosaAI announced a strategic partnership with Broadcom Inc. (AVGO.US) to jointly develop its third generation (next generation) AI inference accelerator, with the goal of sampling in the first half of 2028. It is reported that this cooperation between the two parties is no longer limited to the traditional ASIC cooperation model. The chip will combine 2nm advanced process computing die, independent I/O die, HBM4(E) memory stack, and use Broadcom Inc.'s vertical expansion Ethernet technology to achieve full chip interconnection within the rack, and will be shipped in rack-level system form. This cooperation between the two parties is built on the commercial maturity of RNGD. RNGD is an AI chip launched by FuriosaAI for data center inference scenarios, which is currently in mass production and manufactured using Taiwan Semiconductor Manufacturing Co., Ltd. Sponsored ADR5's 5nm advanced process. RNGD is a 180-watt AI accelerator based on the PCIe interface. FuriosaAI stated that RNGD has been deployed by Samsung SDS and LG AI Research for large language model and intelligent agent AI workloads in standard air-cooled data centers. FuriosaAI stated that its Tensor Contraction Processor (TCP) architecture has been optimized for the mathematical core of AI computation. The chip prioritizes memory access, focusing on high-bandwidth data transfer and large-scale tensor operations, rather than managing thousands of small threads. Broadcom Inc. Semiconductor Solutions Division President Charlie Kawwas stated, "Inference performance is no longer just about raw computing power, but increasingly depends on data reuse and communication efficiency between servers and racks. By combining FuriosaAI's TCP architecture with Broadcom Inc.'s leading XPU technology and IP platform, Ethernet vertical expansion, and switching matrix, we are building a platform that can address critical bottlenecks for large-scale intelligent agent AI." The cooperation with Broadcom Inc. extends FuriosaAI's broader strategic efforts around vertical integration for inference infrastructure. Earlier this year, FuriosaAI positioned the launch of RNGD as part of a larger drive to reduce reliance on NVIDIA Corporation's software ecosystem. CEO June Paik stated, "The challenge we are facing is to replace the CUDA engine with our own software stack." The cooperation with Broadcom Inc. extends this strategy from single-server optimization to rack-level networking and cluster architectures. The two companies position this cooperation as not just chip-level collaboration. The rack-level inference platform will combine FuriosaAI's inference architecture with Broadcom Inc.'s Ethernet architecture, PCIe technology, advanced packaging capabilities, and AI infrastructure IP to enable inference clusters to scale across thousands of nodes. It is worth noting that this cooperation between Broadcom Inc. and FuriosaAI also reflects the broader transformation happening within AI infrastructure, as inference workloads begin to diverge from the training systems that have powered NVIDIA Corporation's rise. While large-scale model training still heavily relies on tightly coupled GPU clusters and proprietary interconnect technologies like NVLink, operators deploying inference infrastructure at production scale are increasingly facing a range of different constraints, including power density, network efficiency, memory bandwidth, latency, and token throughput. Ron Westfall, Vice President and Practice Leader of HyperFrame Research Networks and Infrastructure, stated that large-scale inference is shifting the priority of AI infrastructure away from shaping the needs of GPU-intensive training clusters. He stated, "Large-scale inference shifts bottlenecks to optimizing overall costs, memory bandwidth, and per-token power consumption." Regarding the cooperation between Broadcom Inc. and FuriosaAI, Ron Westfall stated that this collaboration reflects the industry's increasing emphasis on network efficiency as AI deployments scale beyond tightly coupled training systems. He said, "Optimizing network efficiency and rack-level connectivity is now as crucial for inference economics as raw chip performance." In addition, Broadcom Inc.'s Ethernet and PCIe technologies will provide the high-bandwidth rack-level connectivity needed for scaling large inference clusters. This architecture also signifies industry growth in Ethernet-based AI infrastructure, as vendors seek alternatives to proprietary GPU architectures. Broadcom Inc. is increasingly positioning itself as a core supplier of networking, switching, and interconnect infrastructure to support large AI clusters, particularly in the context of hyperscale cloud service providers pursuing custom accelerators and heterogeneous computing environments.