proomt

Search

Search posts, papers, and topics

kv cache

RSS
  1. 1

    DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression

    DeepSeek‑V4.1‑Flash is a 552B‑parameter multimodal Mixture‑of‑Experts LLM that supports up to 1 M‑token contexts while slashing KV‑cache memory to 890 bytes/token (≈¼ of its predecessor) via cross‑layer reuse (CSA2) and FP4 quantisation, plus a SWA‑Bounded Replay scheme that cuts persistent cache to 1/8. The Causal Encoder‑Decoder design halves prefill compute (8B vs 16B active parameters) and th…

    Hugging Face Daily Papersarxiv.org3 minpaperHN12710
  2. 2

    Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches

    Fathom introduces a per-query read depth mechanism for sparse decoding over offloaded KV caches, allowing each query to adaptively decide how many bits of each key channel to read. This method significantly speeds up decoding for large language models with long contexts by reducing host memory traffic, achieving 1.67x faster GPU decoding on Qwen3-8B at one million tokens.

    Hugging Face Daily Papersarxiv.org1 minpaper