DCAI
← 返回全部动态
vLLM 官方博客(网页)规则精选08月22日 00:00

Large-Scale Sharded Weight Transfer with Ray Direct Transport (RDT) in vLLM

We implement a native sharded weight transfer engine in vLLM utilizing Ray Direct Transport (RDT), achieving weight transfer for the Kimi K2 model in BF16 on 48

阅读 vLLM 官方博客(网页) 原文 ↗