proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersPritam Deka1 min readpaperadvanced

From Retrieval to Typed Decisions: Calibrated System One Models from Biomedical Sentence Encoders

Read on arxiv.org

Summary

This paper introduces SBERT2S1, a method to convert biomedical sentence encoders into typed decision models, and BIODECIDE, a new biomedical typed-decision suite. It finds that retrieval training benefits prior-fused residual (PFR) models, cross-head (C) models generally outperform PFR, and the RLCD objective trails cross-entropy due to reward normalization issues.

  • Biomedical sentence encoders can be adapted into typed decision models (SBERT2S1) for schema-constrained questions.
  • Retrieval pre-training significantly helps prior-fused residual (PFR) decision models, but less so for cross-head (C) models.
  • Cross-head (C) decision models consistently outperform prior-fused residual (PFR) models across various objectives.
  • The RLCD training objective's performance deficit compared to cross-entropy stems from its reward normalization.

Engineers working on structured information extraction and decision-making from text, particularly in specialized domains like biomedicine, will find valuable insights into model architecture, pre-training strategies, and objective function design.

8/10
llmmachine learninginformation retrievalnlpmodel trainingbiomedical

Related reading

  1. Introducing System One Models and Jev

    TypeSafe AI announced its first “System One” model, Jev, a non‑text‑generating LLM that outputs type‑safe structured decisions with calibrated probabilities. It claims 40‑200× lower latency (70‑500 ms) and 100‑500× lower cost versus frontier LLMs, no hallucinations, and parallel sampling. The post includes a side‑by‑side demo, a custom “workflow” benchmark comparing Jev to GPT‑5.6/6 and other mod…

    Hacker News front pagetypesafe.ai9 minHN1824480lobste.rs26

  2. Efficient Estimation of Word Representations in Vector Space

    This paper introduces two novel log-linear model architectures for efficiently computing continuous word vector representations from very large datasets. These models achieve state-of-the-art accuracy on syntactic and semantic word similarity tasks with significantly lower computational cost than previous neural network approaches.

    Hall of Famearxiv.org27 minpaper

  4. Reverse-engineered Jev-like model

    Jevlike is an open‑source starter model that scores a list of text options in a single forward pass. It provides a minimal architecture (option queries, shared dot‑product scorer), synthetic data generation, training/evaluation CLI, and examples on Doom and chess. The repo supports a byte‑level encoder or a frozen Hugging‑Face encoder (e.g., Qwen2.5‑0.5B), runs on CPU/MPS/CUDA, and reports benchm…

    Hacker News front pagegithub.com4 minreleaseHN16224

  5. TypeSafe AI's Jev now available on AI Gateway

    Vercel AI Gateway now offers Jev, a probabilistic decision model that returns typed choices, scores, and booleans instead of raw text. TypeSafe AI reports it runs up to 193× faster and 445× cheaper than standard LLMs, exposed via the experimental evaluate API in AI SDK 7.

    Vercelvercel.com2 minrelease

  6. Scaling Laws for Neural Language Models

    This paper empirically studies scaling laws for neural language model performance, finding that cross-entropy loss scales as a power-law with model size, dataset size, and compute. It shows that optimal compute-efficient training involves using very large models, training on relatively modest data, and stopping significantly before convergence.

    Hall of Famearxiv.org67 minpaper