proomt

Search

Search posts, papers, and topics

All posts

Hacker News front pageJinli Hu, Ross M. Clarke, Yichuan Zhang, José Miguel Hernández-Lobato2 min readpaperadvanced

Infinite-Parameter LLMs: Generating and Adapting Weights from Live Data

Read on arxiv.orgHN 157·43 comments

Summary

The paper presents Infinite-Parameter LLMs, where a compact hypernetwork creates feed‑forward weights from live user data and updates a Bayesian latent code online, keeping the stored model size constant while effectively having infinite parameters. This design aims to improve over standard in‑context learning and retrieval by persisting knowledge in weights and freeing context space.

  • A hypernetwork generates feed‑forward weights from live interaction data, turning runtime inputs into model parameters.
  • The generator maintains a Bayesian belief over its latent code, updating it online across a session.
  • Model footprint stays fixed while the effective parameter space becomes unbounded.
  • Weight‑based adaptation frees the prompt context window and persists knowledge across turns.

LLM engineers who need models to adapt continuously from user interactions without full retraining should consider this approach.

5/10
llmmachine learningml infratransformersbayesian

Related reading

  1. Learning to solve hard problems in RL for LLMs by never giving up

    The post introduces the *Matthew Effect* in RL‑fine‑tuning of LLMs—performance gains concentrate on tasks the model already solves— and proposes *Never Give Up* (NGU), an adaptive sampling scheme that uses a small k for easy prompts and retries hard prompts with a high‑probability “never give up” loop. Experiments on math (AIME, GSM8k), code (Manufactoria), and larger‑scale setups (DeepScaler) sh…

    Hacker News front pagegithub.io11 minHN1179

  2. Online Learning with LLM Experts from Limited Feedback

    The paper models prompt routing to multiple LLM experts as a bandit problem with limited feedback and proposes algorithms that achieve sublinear regret in both full‑information and bandit settings. Experiments demonstrate that the methods learn effective routing strategies across diverse LLMs using only a small feedback budget.

    Hugging Face Daily Papersarxiv.org2 minpaper

  3. How LLMs Can Find a Needle in a Haystack

    The post explains how retrieval‑augmented generation (RAG) lets LLM‑based assistants answer questions from private corpora. It covers chunking documents into passages, embedding queries and chunks, similarity metrics, and the trade‑offs of different vector indexes (flat, IVF, HNSW). The focus is on practical design choices rather than new research.

    ByteByteGobytebytego.com12 min

  4. BI-Agent and BI-Bench: Towards Automating End-to-End Business Intelligence

    The paper introduces BI‑Bench, a new benchmark of real‑world BI questions derived from public dashboards, and BI‑Agent, a tool‑augmented LLM system that breaks BI workflows into search, join, and transform subtasks. Baseline LLMs hit <50 % accuracy on BI‑Bench. By orchestrating specialized data‑management tools and post‑training the model with supervised fine‑tuning and reinforcement learning on…

    Hugging Face Daily Papersarxiv.org2 minpaper

  6. LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence

    LimiX-2 is a new tabular foundation model that replaces the usual target‑centric in‑context learning objective with a joint‑distribution objective via Contextual Mechanism Networks (CMNs). Trained on synthetic causal graphs using Context‑Conditional Masked Modeling, it outperforms prior tabular PFNs on TabArena, TALENT, and BCCO and can recover causal skeletons from attention patterns.

    Hugging Face Daily Papersarxiv.org2 minpaper