Hugging Face Daily PapersDeepSeek-AI, Anyi Xu, B. Li3 min readpaperadvanced
DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
Summary
DeepSeek‑V4.1‑Flash is a 552B‑parameter multimodal Mixture‑of‑Experts LLM that supports up to 1 M‑token contexts while slashing KV‑cache memory to 890 bytes/token (≈¼ of its predecessor) via cross‑layer reuse (CSA2) and FP4 quantisation, plus a SWA‑Bounded Replay scheme that cuts persistent cache to 1/8. The Causal Encoder‑Decoder design halves prefill compute (8B vs 16B active parameters) and th…
- KV‑cache compression is tackled on three fronts: (1) cross‑layer reuse in Compressed Sparse Attention 2, (2) 4‑bit floating‑point (FP4) KV storage, and (3) SWA‑Bounded Replay to keep most cache off‑chip.
- Memory footprint drops from ~3.5 KB/token (DeepSeek‑V4‑Flash) to 0.89 KB/token in HBM, and persistent SSD/host memory usage falls to ~1/8 of the baseline.
- CED architecture activates 16 B parameters per token during decode but only 8 B during prefill, improving cost for long‑horizon agents.
- Model trained on 45 T multimodal tokens; evaluation shows better latency‑throughput and task performance despite the smaller cache.
KV‑cache size is the dominant cost driver for serving LLMs with long contexts. By reducing per‑token cache to sub‑kilobyte levels without sacrificing accuracy, DeepSeek‑V4.1‑Flash makes million‑token agents viable on current HBM/SSD budgets, opening the door to cheaper, higher‑throughput deployment…
8/10
