preference learning

Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Modelsarxiv.org

The authors release a 12k‑instance multilingual dilemma dataset (English + Hindi, Arabic, Spanish, Chinese) covering three pairwise value conflicts (Honesty‑Justice, Justice‑Autonomy, Autonomy‑Honesty). Benchmarking GPT‑5‑mini shows a consistent Honesty‑over‑Autonomy bias across languages. Llama‑3.2‑1/3B models exhibit a first‑option bias that can be eliminated (>98% accuracy) via plain fine‑tuni…