Hugging Face Daily PapersNan Li, Albert Gatt, Massimo PoesioSep 16, 20261 min readpaperintermediate

Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of MapTask and MUNDEX

Summary

Cross‑corpus study of gaze behavior in two collaborative dialogue datasets (MapTask, MUNDEX) shows that task‑aligned references correlate with more task‑directed, less partner‑directed gaze, lower entropy and fewer transitions. Temporal gaze features (MapTask) and raw proportion features (MUNDX) modestly improve grounding prediction over baselines, but effects are small and diminish when aggregat…

Mapped both corpora to a unified partner/task/away label set to compare gaze patterns.
Aligned references (MapTask) and UND judgments (MUNDEX) exhibit more task‑focused gaze and reduced partner‑focused gaze.
Gaze entropy drops and transition counts fall at moments of successful grounding.
Temporal gaze features (e.g., fixation windows) give the best lift in MapTask; raw proportion features work best in MUNDEX.

Understanding how visual attention signals grounding can inform multimodal dialogue systems and assistive interfaces that need to infer shared understanding from limited cues.

6/10

nlp dialogue human computer interaction gaze grounding

Related reading

Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental States arxiv.org
Mind2Dialogue introduces a psychology‑guided simulator that generates user mental states and uses privileged distillation to train LLM assistants on Oracle responses that know those states. The resulting models improve personalization and theory‑of‑mind metrics by up to 41 percentage points versus standard instruction‑tuned baselines.
Hugging Face Daily Papersarxiv.orgZixuan Wang, Yufan Zhou, Jinzhou Tang8d2 minpaper
MLLMs Hallucinate when Information Distribution Drifts in Synergy Heads arxiv.org
This paper introduces HEAL, a method to identify and mitigate hallucinations in Multimodal Large Language Models (MLLMs). It finds that hallucinations stem from information distribution drift in "synergy heads" within the attention mechanism and proposes dynamic calibration to steer output towards factual evidence.
Hugging Face Daily Papersarxiv.orgMeng'en Qin, Junye Chen, Jucheng Liu17d1 minpaper
PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection arxiv.org
PANORAMA introduces a VLM that grounds each phrase of a caption to pixel masks by selecting from a phrase‑conditioned pool of mask proposals, trained jointly with caption generation. It sets a new benchmark on the PanoCaps dataset and outperforms prior methods on several pixel‑level grounding tasks.
Hugging Face Daily Papersarxiv.orgSara Pieri, Evangelos Kazakos, Shizhe Chen6d1 minpaper
OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue arxiv.org
OmniVChat defines native audio‑visual dialogue where a model consumes raw audio and video streams and replies in text. The authors build OmniVChat‑Studio, a multi‑agent simulator that generates single‑ and multi‑turn audio‑visual conversations, and use it to create OmniVChat‑Bench, a benchmark covering five dialogue abilities. They also propose OmniVChat‑RL, a reinforcement‑learning reward that b…
Hugging Face Daily Papersarxiv.orgHaolin He, Yunfei Chu, Qi Chen4d2 minpaper
ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models arxiv.org
Action tokenizers for autoregressive VLA models often fail to preserve subtle action adjustments, despite good pointwise reconstruction. This paper introduces Physical Rank Consistency (PRC) to measure relational fidelity and ActionPiece, a new tokenizer that uses joint supervision to preserve these physical relationships, significantly improving policy success on robotics benchmarks.
Hugging Face Daily Papersarxiv.orgShijie Lian, Bin Yu, Zhaolong Shen6d1 minpaper
9 top conversational AI platforms in 2026 twilio.com
Twilio’s blog lists the nine leading conversational‑AI platforms for 2026, highlighting how the market consolidated and what capabilities matter when choosing a vendor. It details Twilio’s own infrastructure tools—ConversationRelay, Agent Connect, Orchestrator, and Memory—showing sub‑second latency and full‑stack reliability for voice and messaging.
Twiliotwilio.comJesse Sumrak7d12 min

Gaze as Evidence for Common Grounding: A Cross-Corpus Analysis of MapTask and MUNDEX

Summary

Related reading

Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental Statesarxiv.org

MLLMs Hallucinate when Information Distribution Drifts in Synergy Headsarxiv.org

PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selectionarxiv.org

OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialoguearxiv.org

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Modelsarxiv.org

9 top conversational AI platforms in 2026twilio.com

Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental States arxiv.org

MLLMs Hallucinate when Information Distribution Drifts in Synergy Heads arxiv.org

PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection arxiv.org

OmniVChat: Synthesizing, Benchmarking, and Training for Native Audio-Visual Dialogue arxiv.org

ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models arxiv.org

9 top conversational AI platforms in 2026 twilio.com