proomt

Search

Search posts, papers, and topics

All posts

Lobsters

Unsizing unsized values

Related reading

  1. How Value Induction Reshapes LLM Behaviour

    Apple researchers fine‑tune LLMs on curated subsets of value‑oriented preference data and measure cross‑value effects, safety, and anthropomorphic language. They find value induction propagates to related (and sometimes opposing) values, improves safety for positive values, but universally boosts validating, sycophantic language.

    Apple Machine Learning Researchapple.com1 minpaper
  2. A Zeroth-Order Paradigm for LLM Preference Alignment

    The paper proposes Comparison-based Preference Optimization (ComPO), a zeroth‑order method that uses comparison oracles to align LLMs without a differentiable loss. Experiments on several LLM families show it improves win rates and mitigates likelihood displacement compared to direct alignment approaches.

    Hugging Face Daily Papersarxiv.org1 minpaper
  3. VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention

    VC-Attention introduces a training‑free low‑bit attention pipeline for diffusion transformers. It smooths value tensors via lightweight online clustering (V‑Smooth) and quantizes only the residual after subtracting block means, restoring the mean from the softmax row sum. It also replaces the FP32 softmax exponential with a fused FP8 cast (ExpCast‑FP8) that maps log‑scores directly to E4M3 probab…

    Hugging Face Daily Papersarxiv.org1 minpaper
  4. Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem

    The authors cast transformer block removal as a constrained binary optimization problem equivalent to an Ising glass, using a Hessian‑derived energy as a proxy for downstream quality. Solving the resulting QUBO with classical or quantum‑inspired solvers yields up to 23 MMLU points improvement over prior block‑removal baselines at 50 % depth compression.

    Hugging Facehuggingface.co8 min