PDD Architecture Breaks Cross-Cluster Heterogeneous Inference Barrier: 51.5% Lower First-Token Latency, 37.5% Cost Reduction
InfiniteG (Infinigence) unveiled PDD, a three-layer cross-cluster heterogeneous inference architecture that reduces first-token latency by 51.5% and per-token cost by 37.5%. By inserting a local RelayDecode (RLD) instance to mask WAN KV Cache transfer delays, PDD enables efficient utilization of dispersed homogeneous clusters for LLM inference.
Infinigence
QbitAI 量子位30.07 · 11:02