Hugging Face Daily PapersUtkarsh Agarwal, Monojit Choudhury1 min readpaperadvanced
Geometry of Values: Task Vector Composition for Ethical Preference Alignment in Language Models
Summary
The authors release a 12k‑instance multilingual dilemma dataset (English + Hindi, Arabic, Spanish, Chinese) covering three pairwise value conflicts (Honesty‑Justice, Justice‑Autonomy, Autonomy‑Honesty). Benchmarking GPT‑5‑mini shows a consistent Honesty‑over‑Autonomy bias across languages. Llama‑3.2‑1/3B models exhibit a first‑option bias that can be eliminated (>98% accuracy) via plain fine‑tuni…
- A new 12 k‑example, 4‑language moral dilemma benchmark for probing value conflicts in LLMs.
- Cross‑lingual evaluation reveals systematic Honesty‑over‑Autonomy bias in GPT‑5‑mini.
- Fine‑tuning (standard or DPO) removes first‑option bias in Llama‑3.2‑1/3B, achieving >98 % correct alignment on the benchmark.
- Proposes a task‑vector method: compute a value‑preference direction, orthogonalize to the instruction‑following vector, then add/subtract to control stance.
Aligning LLMs to nuanced, often conflicting human values is a core safety challenge. This work provides both a multilingual benchmark to surface hidden biases and a concrete, vector‑space technique to isolate and edit specific value directions, offering a scalable tool for ethical fine‑tuning beyon…
7/10
