proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersMaëlic Neau2 min readpaperadvanced

RelateAnything: Real-Time Open-Vocabulary Relation Prediction From Any Inputs

Read on arxiv.org

Summary

RelateAnything is a 53 M‑parameter model that predicts arbitrary textual relations between any supplied image regions, using a text‑embedding bank instead of a fixed classifier. Trained on a new 4.3 M‑relation dataset, it outperforms prior open‑vocab methods by 2.3‑3.5× mean recall while running in ~20 ms per frame.

  • Model accepts region proposals from any source (detector, human, etc.) and a predicate list supplied at inference as raw strings.
  • Predicate vocabulary is encoded with a contrastive text encoder; antonyms are separated at cosine similarity ≥ 0.95.
  • Training uses positive‑unlabeled supervision over 19 k predicates, leveraging the RA‑4M dataset (474 k images, 4.3 M relations).
  • OV‑SGG‑Bench evaluates on six axes across multiple datasets, revealing that in‑domain scores can overstate transfer gains by ~5×.

Vision‑language engineers who need flexible, real‑time relation extraction across arbitrary vocabularies will benefit from the model’s low latency and dataset‑driven approach.

8/10
vision languagecomputer visiontransformersscene graphopen vocabulary

Related reading

  1. LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence

    LimiX-2 is a new tabular foundation model that replaces the usual target‑centric in‑context learning objective with a joint‑distribution objective via Contextual Mechanism Networks (CMNs). Trained on synthetic causal graphs using Context‑Conditional Masked Modeling, it outperforms prior tabular PFNs on TabArena, TALENT, and BCCO and can recover causal skeletons from attention patterns.

    Hugging Face Daily Papersarxiv.org2 minpaper

  2. Zing-0.5: Toward Playable Worlds with Real-Time Joint Action and Text Control

    Zing‑0.5 is a 5 B autoregressive world model that lets users control generated environments in real time using both keyboard actions and text prompts. The paper introduces unified action‑text conditioning, segment‑level teacher distillation, and a low‑cost streaming inference pipeline that runs at 24 FPS (832×480) for about $0.009 per minute, achieving 81 % overall and 88.5 % consistency on a nav…

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. JEPA-Anything: Learning Predictive Models across Different Worlds

    JEPA-Anything extends joint‑embedding predictive architectures with orthogonal predictive factorization, letting a single model learn complementary latent factors that can be recombined for prediction across disparate domains. The paper shows consistent performance gains on ten dynamics tasks, molecular simulations, and clinical event forecasting, plus experimental validation of a biologically‑de…

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. ActionPiece: Rethinking Action Tokenization for Autoregressive Vision-Language-Action Models

    Action tokenizers for autoregressive VLA models often fail to preserve subtle action adjustments, despite good pointwise reconstruction. This paper introduces Physical Rank Consistency (PRC) to measure relational fidelity and ActionPiece, a new tokenizer that uses joint supervision to preserve these physical relationships, significantly improving policy success on robotics benchmarks.

    Hugging Face Daily Papersarxiv.org1 minpaper