DCAI
← 返回全部动态
vLLM 官方博客(网页)规则精选09月22日 00:00

Announcing vllm-metal: Concurrent Serving on Apple Silicon

vllm-metal brings vLLM's paged, continuously batched serving stack to Apple Silicon, with flatter TTFT under concurrent agent load, batched MTP, and automatic M

阅读 vLLM 官方博客(网页) 原文 ↗