Hugging Face Daily PapersRanran Haoran Zhang, Aysa Xuemo Fan, David Munhá Correia, Alex Cheema, Rui Zhang1 min readpaperadvanced
SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops
Summary
SiliconBench benchmarks nine Apple‑Silicon LLM serving engines on unified‑memory desktops, measuring speed, memory usage, and output fidelity across Qwen3, Qwen3.5, and Gemma‑4 models. It finds vllm‑metal doubles throughput at modest concurrency, memory budgets often fail to preserve headroom, and only three stacks satisfy all fidelity and model‑coverage requirements, with tensor‑parallel scaling…
- vllm‑metal on Apple Silicon more than doubles throughput for Qwen3‑0.6B when scaling concurrency from 1 to 16, while CUDA vLLM and SGLang show stronger scaling at higher concurrency.
- Explicit memory budgets do not guarantee headroom; two stacks hit physical memory limits under load, causing throughput to drop.
- Only three of the nine evaluated stacks meet all three gates (completion, fidelity, model coverage) for the tested models.
- Packed prefill‑decode path in vllm‑metal yields lower first‑token latency than omlx under concurrent workloads.
Engineers building or optimizing local LLM serving on Apple Silicon desktops need realistic performance, memory, and fidelity data to choose the right stack and scaling strategy.
7/10
