Hugging Face Daily PapersChuhao Chen, Peter Wonka, Chaoyang Wang1 min readpaperadvanced
PhysStream: Streaming Physics-Grounded Video Generation with Structured Scene Memory and Fine-Grained Motion Control
Summary
PhysStream introduces a two‑stage autoregressive video generator that uses online‑derived positional and tracking maps (structured scene memory) and sparse velocity‑increment signals to enable fine‑grained, physics‑grounded control of multi‑object tabletop scenes. It cuts motion distribution error by 33 % and trajectory error by 12 % versus strong baselines, and wins 85 % of human preference test…
- Structured scene memory (positional + tracking maps) lets a generative model maintain physical consistency across frames without needing the full control schedule up‑front.
- Sparse velocity‑increment conditioning encodes physical quantities directly, giving users fine‑grained, interpretable motion control.
- A two‑stage training pipeline—bidirectional fine‑tuning for motion conditioning followed by causal autoregressive training with memory—yields measurable gains in dynamics fidelity.
- Benchmarks on synthetic tabletop rigid‑body setups show substantial reductions in FVMD and trajectory error, and human evaluators strongly prefer the outputs.
Interactive video synthesis has been stuck at coarse prompt‑level control or pixel‑space nudges that ignore underlying physics. PhysStream demonstrates that embedding explicit physical signals and scene‑level memory into the generation loop can produce controllable, physically plausible motion, ope…
8/10
