Related reading
Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches
Fathom introduces a per-query read depth mechanism for sparse decoding over offloaded KV caches, allowing each query to adaptively decide how many bits of each key channel to read. This method significantly speeds up decoding for large language models with long contexts by reducing host memory traffic, achieving 1.67x faster GPU decoding on Qwen3-8B at one million tokens.
Hugging Face Daily Papersarxiv.org1 minpaperDeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression
DeepSeek‑V4.1‑Flash is a 552B‑parameter multimodal Mixture‑of‑Experts LLM that supports up to 1 M‑token contexts while slashing KV‑cache memory to 890 bytes/token (≈¼ of its predecessor) via cross‑layer reuse (CSA2) and FP4 quantisation, plus a SWA‑Bounded Replay scheme that cuts persistent cache to 1/8. The Causal Encoder‑Decoder design halves prefill compute (8B vs 16B active parameters) and th…
Hugging Face Daily Papersarxiv.org3 minpaperHN12710tokenizers v1: encode, decode and scaling, measured
Hugging Face has released `tokenizers` v1, a major performance update that achieves 3-30x faster encoding than v0.23 while maintaining identical output and API compatibility. Key optimizations include a SIMD-accelerated splitter, a thread-local word cache, and an allocation-free BPE merge loop, ensuring tokenization doesn't bottleneck ML workflows.
Hugging Facehuggingface.co10 minArticle: Your Next DSL Author Is a Language Model
Typed Domain Grounding (TDG) embeds a DSL inside a mainstream language the LLM already knows (e.g., Kotlin) and uses the host compiler as an oracle. The author describes five building blocks—embedding, choosing a host language with high training‑data frequency, compiler‑driven type safety, a generate‑compile‑repair loop, and an on‑demand teaching tool—and shows measured results from kUML, a Kotli…
InfoQinfoq.com18 minMoME: Mixture-of-Memory Embeddings for Context-Aware Sparse Lookup
The paper proposes Mixture of Memory Embeddings (MoME), a context‑aware sparse lookup that replaces each token’s single memory row with a gated mixture of multiple slots. Experiments on Llama‑3, MobileLLM and Qwen3 show MoME outperforms existing memory‑embedding baselines at equal parameter and FLOP budgets and exhibits interpretable routing for polysemous tokens.
Hugging Face Daily Papersarxiv.org1 minpaperAccelerating the borderless Lakehouse: Announcing preview of cross-cloud caching
Google Cloud previewed cross‑cloud caching for its Borderless Lakehouse. The feature caches sub‑file Parquet blocks in Google Cloud, encrypts them with GMEK, isolates cache per tenant/region, and validates freshness via metadata checks. In tests it can reduce cross‑cloud data transfer to <5% of the original size, lowering query latency and cost for Iceberg tables stored in other clouds. BigQuery…
Google Cloud Bloggoogle.com3 minrelease
