proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersYujia Hu, Jiajun Li, Zihao He1 min readpaperadvanced

Streaming Video Editing with Easy Adaptation

Read on arxiv.org

Summary

The paper introduces SVEET, a framework that adapts a pretrained bidirectional video diffusion model for streaming video editing via an auxiliary branch with temporally independent self‑attention and a decoupled orthogonal training scheme. It achieves real‑time 15 FPS editing on a single H100 GPU without extra acceleration.

  • SVEET adds an auxiliary branch that encodes source frames with temporally independent self‑attention and injects features into the backbone for streaming control.
  • Training enforces orthogonality between controllability and causality gradients, allowing zero‑shot transfer across different backbone architectures.
  • The method runs at ~15 FPS on a single H100 GPU, matching real‑time requirements without specialized hardware tricks.
  • Only a pretrained bidirectional video diffusion model is needed; no full retraining of the backbone.

Engineers building real‑time generative video editors or streaming pipelines need efficient, adaptable diffusion models, and SVEET offers a practical path.

6/10
computer visiongpugenerative modelsstreamingvideo diffusion

Related reading

  1. Video DeltaNet: A Video-Native Hybrid Attention for Livestream Video Generation

    Video DeltaNet (VDN) replaces full‑softmax attention in video diffusion models with a hybrid: per‑frame local Softmax for fine detail and a bidirectional linear memory (Video Delta Attention) for long‑range context. A teacher‑alignment schedule injects the linear branch into a pretrained MiniMax H3 model, preserving Softmax for text/audio streams. On eight NVIDIA B200 GPUs VDN‑H3 denoises a 14.3‑…

    Hugging Face Daily Papersarxiv.org1 minpaper

  2. Why Do Video Diffusion Models Violate Physics? Unveiling the Flaws in Attention Mechanisms

    The paper investigates why video diffusion models often break physical laws, pinpointing excessive spatial decay from Rotary Position Embedding (RoPE) as the culprit. By analyzing cross‑attention trajectories and self‑attention patterns, the authors identify specific attention heads that drive motion planning. They propose a lightweight fix: scaling RoPE frequency per denoising step, which empiri…

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs

    Flash-dLLM is a training-free framework that accelerates Diffusion LLM inference by addressing GPU memory I/O bottlenecks with an I/O-aware KV-cache kernel. It also introduces a KV-cache-driven draft-and-verify decoding strategy, achieving significant speedups (up to 11x) over prior methods.

    Hugging Face Daily Papersarxiv.org1 minpaper

  4. PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control

    PhysStream introduces a two‑stage autoregressive video generator that uses online‑derived positional and tracking maps (structured scene memory) and sparse velocity‑increment signals to enable fine‑grained, physics‑grounded control of multi‑object tabletop scenes. It cuts motion distribution error by 33 % and trajectory error by 12 % versus strong baselines, and wins 85 % of human preference test…

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control

    Zing‑0.5 is a 5 B autoregressive world model that lets users control generated environments in real time using both keyboard actions and text prompts. The paper introduces unified action‑text conditioning, segment‑level teacher distillation, and a low‑cost streaming inference pipeline that runs at 24 FPS (832×480) for about $0.009 per minute, achieving 81 % overall and 88.5 % consistency on a nav…

    Hugging Face Daily Papersarxiv.org1 minpaper