ZCube - The next-generation large model inference network architecture jointly launched by Zhipu AI and Tsinghua University

What is ZCube

ZCube is a next-generation large model inference network architecture jointly launched by Zhipu AI, Yuxun Networks, and Tsinghua University, designed specifically to address structural network congestion in PD-disaggregated deployment. The architecture eliminates traditional Spine-layer switches, adopts a fully flattened topology and a single/multi-rail hybrid access mechanism, and achieves global traffic decoupling and discretized routing. In production testing in the GLM-5.1 coding environment, ZCube reduced switch and optical module capital expenditure by 33%, increased average GPU inference throughput by 15%, and reduced TTFT P99 by 40.6%, while keeping the GPU and software stack unchanged.

Main features of ZCube

• Flattened networking with the Spine layer removed: It breaks away from the traditional hierarchical stacking approach of the Clos architecture, retains the Leaf switch layer, and interconnects it through a complete bipartite graph, compressing the network diameter from 3 hops to 2 hops.
• Single/multi-rail hybrid access mechanism: One group of Leaf switches connects to consecutively numbered GPUs in single-rail mode, while another group connects to identically numbered GPUs in multi-rail mode, naturally discretizing the dynamic asymmetric traffic generated by PD disaggregation.
• Global load-balanced routing: The ZCube routing policy ensures that only one unique optimal path exists between any pair of GPUs, avoiding traffic conflicts caused by multipath route selection and achieving ideal global load balancing across the entire network in both training and inference scenarios.
• Structural congestion elimination: Through architectural innovation, KV Cache cross-node transmission traffic is globally decoupled and discretized, fundamentally eliminating local hotspot links and PFC backpressure.

Technical principles of ZCube

• Complete bipartite graph topology: Leaf switches are divided into two groups by odd and even numbers, switches within a group are not connected to each other, and each switch in one group is fully connected to all switches in the other group, forming a flattened network.
• Dual-port differentiated access: The two ports of each GPU NIC are connected in single-rail mode (connecting to odd-numbered switches) and multi-rail mode (connecting to even-numbered switches), respectively, using mathematical mapping formulas to achieve deterministic routing.
• Deterministic shortest-path routing: Based on modular arithmetic and ceiling mapping between GPU numbers and switch numbers, it ensures that there is only one optimal path passing through two Leaf switches between any two GPUs.
• Traffic pattern adaptation: For the source-destination asymmetric and dynamically changing KV Cache transmission characteristics generated by PD disaggregation, the topology itself naturally disperses traffic, without relying on transport-layer mechanisms such as adaptive routing or packet spraying.

How to use ZCube

• Architecture planning: Divide Leaf switches into two groups by odd and even numbers, design a complete bipartite graph interconnection topology, and remove the traditional Spine layer.
• Port access configuration: Configure a single-rail and multi-rail hybrid access policy for the two ports of each GPU NIC, respectively, connecting them to the corresponding odd and even switches.
• Automated deployment: Use the ZCube controller, data center layout design tool, and cabling correctness verification program to complete automatic configuration generation and batch delivery.
• Production verification: Before the inference service goes live, monitor bandwidth, latency, and PFC events, and formally switch traffic after confirming there are no structural congestion hotspots.

Core advantages of ZCube

• Significantly lower cost: At the same scale, it reduces switch and optical module investment by about 33% compared with Clos/ROFT, and a 10,000-GPU intelligent computing cluster can save about RMB 210 million to RMB 640 million in network hardware investment.
• Inference performance improvement: In the GLM-5.1 coding production test, average GPU inference throughput increased by more than 15%, and the P99 percentile of TTFT decreased by 40.6%.
• Ultra-strong scalability: Based on existing 51.2T switches, a flat network connecting 16,384 400Gbps NICs can be built, and through multi-plane partitioning it can support interconnection of tens of thousands to hundreds of thousands of GPUs.
• Zero-intrusion upgrade: No changes are required to GPU hardware, software stack, or application logic; existing hardware potential can be unleashed solely through tuning at the network architecture layer.

ZCube project address

• Project official website：https://z.ai/blog/zcube

Comparison of ZCube with similar competing products

ZCube application scenarios

• Ultra-large-scale LLM inference clusters: Suitable for thousand-card to ten-thousand-card level inference clusters with PD-disaggregated deployment, solving the network bottleneck caused by cross-node KV Cache transmission.
• Long-context inference services: The architecture can effectively alleviate the constraints imposed by network bandwidth on first-token latency and overall throughput in long-sequence scenarios.
• High-density intelligent computing centers: Provides a highly balanced, low-latency, and high-bandwidth-utilization network foundation for mixed training and inference workloads.
• MaaS cloud service platforms: The architecture can reduce the overall cost of inference services and improve tail-latency stability in multi-tenant high-concurrency scenarios.