proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersHaocheng Xi, Yiming Xie, Hexu Zhao1 min readpaperintermediate

Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation

Read on arxiv.org

Summary

Video DeltaNet (VDN) replaces full‑softmax attention in video diffusion models with a hybrid: per‑frame local Softmax for fine detail and a bidirectional linear memory (Video Delta Attention) for long‑range context. A teacher‑alignment schedule injects the linear branch into a pretrained MiniMax H3 model, preserving Softmax for text/audio streams. On eight NVIDIA B200 GPUs VDN‑H3 denoises a 14.3‑…

  • Full‑softmax attention is the main compute cost in video diffusion denoising.
  • Linear attention alone loses fine‑grained spatial interactions; a hybrid preserves them.
  • Video Delta Attention updates a per‑frame linear memory by aggregating all spatial tokens, enabling bidirectional long‑range context with O(N) cost per frame.
  • Separate output projections and learnable gates balance the two branches.

Inference cost is a bottleneck for real‑time or low‑latency video generation (e.g., livestream overlays, interactive avatars). By cutting attention complexity from quadratic to linear while retaining high‑fidelity detail, VDN makes diffusion‑based video generation feasible at near‑real‑time speeds…

7/10
machine learningdiffusionperformanceml infravideoattention

Related reading

  1. SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization

    This paper introduces SpectralShift, a spectral reparameterization method for extending the context window of Gated DeltaNet (GDN) linear attention models. It reconfigures the decay spectrum by enhancing slow propagation and preserving fast-decaying modes, consistently improving long-context capabilities during continual pretraining.

    Hugging Face Daily Papersarxiv.org1 minpaper

  2. VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention

    VC-Attention introduces a training‑free low‑bit attention pipeline for diffusion transformers. It smooths value tensors via lightweight online clustering (V‑Smooth) and quantizes only the residual after subtracting block means, restoring the mean from the softmax row sum. It also replaces the FP32 softmax exponential with a fused FP8 cast (ExpCast‑FP8) that maps log‑scores directly to E4M3 probab…

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation

    OmniVBench is a new benchmark and the Omni‑R2V Dataset, offering 7 task families, 18 fine‑grained reference‑to‑video generation tasks and a factor‑grounded evaluation checklist of over 12 k items. The dataset provides 340 k industrial‑grade video samples and pipelines for constructing reference‑target pairs, exposing large performance gaps in current R2V models.

    Hugging Face Daily Papersarxiv.org2 minpaper

  4. PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control

    PhysStream introduces a two‑stage autoregressive video generator that uses online‑derived positional and tracking maps (structured scene memory) and sparse velocity‑increment signals to enable fine‑grained, physics‑grounded control of multi‑object tabletop scenes. It cuts motion distribution error by 33 % and trajectory error by 12 % versus strong baselines, and wins 85 % of human preference test…

    Hugging Face Daily Papersarxiv.org1 minpaper

  5. Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control

    Zing‑0.5 is a 5 B autoregressive world model that lets users control generated environments in real time using both keyboard actions and text prompts. The paper introduces unified action‑text conditioning, segment‑level teacher distillation, and a low‑cost streaming inference pipeline that runs at 24 FPS (832×480) for about $0.009 per minute, achieving 81 % overall and 88.5 % consistency on a nav…

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. NVIDIA Vera Rubin NVL72 Delivers Leading Performance in MLPerf Inference v6.1 Debut

    NVIDIA’s Vera Rubin NVL72 AI inference system shows up to 3.7× higher throughput than the prior GB300 NVL72 on MLPerf v6.1 benchmarks (Qwen3‑VL, DeepSeek‑R1), achieves 99% scaling efficiency across 288 GPUs, and benefits from software optimizations (NVFP4 precision, kernel fusion, disaggregated serving). The post is a product announcement with concrete benchmark numbers but limited technical dept…

    Nvidianvidia.com4 min