Hugging Face Daily PapersRuixiang Zhao, Hualei Wang, Renhe Sun1 min readpaperintermediate
Realtime-Venus: A full-duplex interaction system with asynchronous delegation
Summary
Realtime-Venus is a full‑duplex, multimodal dialogue system built from two separately trained 9B models (Omni for audio‑visual, Audio for spoken interaction). It uses a shared causal timeline and a dual‑loop runtime that lets foreground interaction continue while a background Harness executes delegated tasks asynchronously. The paper reports benchmark scores where Realtime‑Venus‑Omni leads on six…
- Two 9B LLMs are trained for distinct modalities but share a unified post‑training recipe (offline understanding + proactive full‑duplex trajectories + delegation workflow).
- A dual‑loop runtime separates foreground interaction from background reasoning, enabling asynchronous tool execution without breaking the dialogue flow.
- Benchmark results show state‑of‑the‑art performance on both video and audio tasks, and strong interruption‑handling metrics (75% response to user interruptions, continuation rates up to 97%).
Real‑time multimodal agents need to handle continuous perception, user interruptions, and background tool use without latency spikes. Realtime‑Venus demonstrates a practical architecture that achieves this with large models, offering a reference for building production‑grade, full‑duplex AI assista…
6/10
