Hugging Face Daily PapersWangbo Yu, Kunhao Liu, Wenbo Hu1 min readpaperadvanced
WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory
Summary
WorldCrafter introduces a camera‑queryable implicit 3D‑aware memory that compresses multi‑view evidence into a fixed token budget for video generation, improving long‑horizon consistency and camera‑control accuracy in streamed scene exploration.
- Memory encoder and pose‑conditioned readout turn past frames into a small set of view‑specific tokens before the diffusion denoiser, avoiding explicit depth correspondences.
- Joint training of memory, encoder, and video generator lets the requested viewpoint dictate how historical evidence is summarized.
- Few‑step distillation and recent temporal context are added to the token set to preserve visual fidelity over minute‑scale generation.
- Benchmarks on static and dynamic scenes report substantial improvements in consistency and camera‑control metrics while keeping image quality comparable to baseline diffusion video models.
Long‑horizon video world models typically drift or lose spatial coherence when the camera moves far from earlier observations. By making the memory queryable by camera pose, WorldCrafter keeps the generated world anchored to all prior views without blowing up token counts, which is a practical step…
8/10