Hugging Face Daily PapersXingxuan Zhang, Gang Ren, Hao Yuan2 min readpaperadvanced
LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence
Summary
LimiX-2 is a new tabular foundation model that replaces the usual target‑centric in‑context learning objective with a joint‑distribution objective via Contextual Mechanism Networks (CMNs). Trained on synthetic causal graphs using Context‑Conditional Masked Modeling, it outperforms prior tabular PFNs on TabArena, TALENT, and BCCO and can recover causal skeletons from attention patterns.
- CMNs shift the learning goal from p(y|x,context) to p(x,y|context), enabling the model to capture underlying generative mechanisms.
- Pre‑training uses large‑scale synthetic datasets generated from diverse structural causal models, following scaling laws derived in earlier LimiX work.
- Evaluation shows state‑of‑the‑art predictive performance on several tabular benchmarks and emergent causal‑awareness (attention aligns with direct causal edges).
Tabular data is ubiquitous in industry, yet most models treat it as a pure prediction problem. By learning the joint distribution conditioned on context, LimiX‑2 can both predict and expose causal structure, opening pathways for more reliable decision‑support systems and for downstream tasks like c…
8/10
