Search posts, papers, and topics
131 posts · huggingface.co
The paper studies long‑horizon memorization where a language model must learn 100 tasks via continual fine‑tuning without retaining data or task IDs. By composing data, function, and weight anchors with merged LoRA, they boost final retention from 1.2% to 34.9%, a 28× improvement.
This paper introduces HEAL, a method to identify and mitigate hallucinations in Multimodal Large Language Models (MLLMs). It finds that hallucinations stem from information distribution drift in "synergy heads" within the attention mechanism and proposes dynamic calibration to steer output towards factual evidence.
Beyond Top‑k Skill Retrieval: Diversity‑Aware Skill Routing (DSR) applies a Determinantal Point Process with a query‑residual diversity kernel to rerank skill candidates, balancing relevance and redundancy. On the SkillRouter benchmark it raises recall and full‑coverage, especially for multi‑skill queries, showing that skill routing benefits from set‑selection rather than independent ranking.
The paper models prompt routing to multiple LLM experts as a bandit problem with limited feedback and proposes algorithms that achieve sublinear regret in both full‑information and bandit settings. Experiments demonstrate that the methods learn effective routing strategies across diverse LLMs using only a small feedback budget.
The paper introduces DataFlex‑RL, a platform to benchmark how different data‑selection policies affect reinforcement‑learning‑with‑verifiable‑rewards training. Across extensive experiments on Qwen2.5‑7B and Llama‑3.1‑8B, uniform sampling is the only method that consistently improves performance, and no alternative policy yields a statistically significant gain.
Code2Skill is an automated pipeline that mines popular GitHub repositories to extract verifiable, implementation‑anchored procedural “skills”. It builds a bank of ~1 M skill records (atomic ops, workflows, patterns) with provenance metadata, verifies each via blind reconstruction, and shows that augmenting LLM‑based agents with these skills yields an average 11.7% performance lift across 72 proto…
Srijika is a system for restyling OpenType fonts across nine Indic scripts by reusing existing template font layout data (GSUB, GPOS). This approach ensures complex script features like conjuncts remain consistent, addressing a major challenge in Indic font generation.
Researchers train a compact 82 M‑parameter Thai fixed‑voice TTS model using synthetic speech generated by a large voice‑cloning teacher, requiring only a 15‑second real reference. The student achieves 68.2% keyword accuracy and 91.4% pause precision, outperforming its teacher on pause placement and enabling on‑device inference.
The paper dissects which aspects of synthetic Thai OCR data (typeface diversity, 2‑D layout, real handwriting glyphs) actually help transfer to real documents. Using those insights they train Wayu-Paxa-OCR-Zero without any real OCR labels and achieve character error rates comparable to or better than supervised baselines.
The paper presents TREVIS, a method that encodes decision trees into a continuous latent space using a Tree Transformer VAE, allowing gradient-based optimization of both accuracy and structural sparsity. Experiments claim TREVIS matches the predictive performance of near-optimal algorithms while producing sparser trees.
PLC‑DPO extends Direct Preference Optimization by using the policy‑reference margin to route each training pair into clean, flipped, or tie categories, actively correcting noisy or ambiguous labels. Across extensive benchmarks it improves mean win‑rate from 55.5 % to 60.5 % and stays stable under injected noise and tie stress tests.
