Hugging Face Daily PapersHaoyu Zhao, Zihao Zhao, Tianyu DengSep 16, 20262 min readpaperadvanced

Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model

Summary

This paper introduces a novel evaluation framework to assess the physical world reasoning capabilities of omni-modal generative models like MiniMax-H3. It found that MiniMax-H3 achieved an overall success rate of 41.97% across 517 instances, with significant performance variations depending on the input modalities and reasoning tasks.

A new evaluation framework is proposed to test omni-modal models on physical world reasoning, leveraging complementary multimodal inputs.
The framework includes four scenarios: implicit prompts, audio-image, prefix-videos, and audio-video inputs, requiring joint reasoning.
MiniMax-H3 achieved a 41.97% overall success rate on 517 evaluation instances.
Video-based Decision Reasoning was the strongest task for MiniMax-H3 (56.00% success rate).

Researchers and engineers developing advanced generative AI models should care, as this work provides a new methodology and specific benchmarks for assessing multimodal reasoning capabilities.

7/10

llm machine learning evaluation multimodal computer vision

Related reading

UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation arxiv.org
UFO introduces an Atomized Chain‑of‑Evaluation (AEU) framework that breaks omni‑condition alignment in multi‑modal image generation into a sequential set of fine‑grained checks, achieving a 15.25 % boost in correlation with human judgments. The authors also release UFO‑Bench, a benchmark for testing how well models satisfy combined textual and visual conditions.
Hugging Face Daily Papersarxiv.orgDanning Zhang, Yijing Lin, Shuhan Zhuang5d1 minpaper
OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning arxiv.org
OmniHarness introduces a symbolic‑policy framework that extracts reusable procedural knowledge from multimodal LLM‑driven visual generation runs. By decoupling task logic from instance inputs, the system can instantiate, adapt, and compose policies for new visual tasks, using intermediate verification for on‑the‑fly refinement while keeping the underlying model frozen. Self‑directed practice task…
Hugging Face Daily Papersarxiv.orgXu Xu, Jinxiu Liu, Zhangbo Qiao9d1 minpaper
Modality-Autoregressive World-Action Models arxiv.org
ModAR is a world‑action model that autoregressively denoises multiple future modalities (depth, DINO features, point tracks) before predicting actions, letting each prediction condition on earlier outputs. It outperforms prior WAMs, achieving higher success rates with ~20× fewer training FLOPs and no pretraining.
Hugging Face Daily Papersarxiv.orgAdam Hung, Bardienus P. Duisterhof, Deva Ramanan7d1 minpaper
Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models arxiv.org
The paper reframes fine‑tuning of instruction‑tuned LLMs as a direction‑selection problem under a fixed behavioral‑drift budget, showing that the update direction, not magnitude, determines trade‑offs between target performance and capability preservation. In QA‑only fine‑tuning of Qwen‑3 models, layer‑selective probing finds effective directions that boost scientific reasoning and multilingual t…
Hugging Face Daily Papersarxiv.orgFei Yuan, Changjiang Gao, Yilei Tu10d1 minpaper
Thought without systematicity? Evaluating reasoning models on rule induction tasks arxiv.org
The paper builds compositional rule‑induction task families and generates isomorphic variants to probe systematicity in reasoning models. Experiments reveal that models frequently fail on equivalent variants despite solving the original task, indicating a lack of systematic reasoning.
Hugging Face Daily Papersarxiv.orgSimon Schug, Brenden M. Lake10d1 minpaper
GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning arxiv.org
GAVEL augments LLM‑driven robot planners with an explicit graph world model that verifies actions, repairs violations, and reasons over belief distributions, boosting single‑task success from 41 % to 92 % and multi‑task success from 20 % to 93 % on BEHAVIOR‑1K.
Hugging Face Daily Papersarxiv.orgRuiyang Wang, Hao-Lun Hsu, Swarajh Mehta6d1 minpaper

Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model

Summary

Related reading

UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generationarxiv.org

OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learningarxiv.org

Modality-Autoregressive World-Action Modelsarxiv.org

Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Modelsarxiv.org

Thought without systematicity? Evaluating reasoning models on rule induction tasksarxiv.org

GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planningarxiv.org

UFO: Chain-of-Evaluation for Omni-Condition Alignment in Multi-Modal Image Generation arxiv.org

OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning arxiv.org

Modality-Autoregressive World-Action Models arxiv.org

Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models arxiv.org

Thought without systematicity? Evaluating reasoning models on rule induction tasks arxiv.org

GAVEL: Graph World Models for Verified and Efficient Long-Horizon LLM Task Planning arxiv.org