proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersAashiq Muhamed, Mona T. Diab, Virginia Smith1 min readpaperadvanced

Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration

Read on arxiv.org

Summary

Decoy Direction Optimization (DDO) is a post‑hoc weight‑editing defense for open‑weight LLMs that injects a high‑magnitude nonlinear decoy into MLP neurons, corrupting contrastive estimators used by Refusal Feature Ablation (RFA) attacks. The paper proves a spectral bound on the effect, evaluates DDO on six model families (including Llama‑3‑8B‑Instruct), and shows <10 % attack success rate (ASR)…

  • Injecting a strong, nonlinear decoy signal can mislead RFA’s contrastive direction estimator, forcing the attacker to ablate an orthogonal, harmless feature.
  • DDO achieves comparable protection to finetuned defenses with orders‑of‑magnitude lower computational cost, making it practical for rapid deployment on new checkpoints.
  • The defense is evaluated across multiple model families and under adaptive multi‑phase attacks, demonstrating robustness but still leaving a non‑zero ASR (~10‑15 %).
  • A spectral bound is provided to formalize how the decoy perturbs the estimator, offering a theoretical guarantee of the defense’s effect.

Open‑weight LLMs are increasingly deployed with safety guardrails that can be bypassed via lightweight post‑hoc attacks like RFA, which avoid costly finetuning. DDO offers a scalable, low‑overhead mitigation that can be applied after model release, addressing a practical security gap in the LLM eco…

8/10
llmmachine learningsecurityml infra

Related reading

  1. Drift-Constrained Optimization: Only Direction Matters in Fine-Tuning Instruct Models

    The paper reframes fine‑tuning of instruction‑tuned LLMs as a direction‑selection problem under a fixed behavioral‑drift budget, showing that the update direction, not magnitude, determines trade‑offs between target performance and capability preservation. In QA‑only fine‑tuning of Qwen‑3 models, layer‑selective probing finds effective directions that boost scientific reasoning and multilingual t…

    Hugging Face Daily Papersarxiv.org1 minpaper

  2. How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus

    This paper investigates the "lossless" claim of Orthrus, a hybrid architecture for accelerating LLM inference. It finds that under BF16 precision, Orthrus diverges from the exact autoregressive output trajectory in over 50% of cases, though FP32 maintains exact matching. Despite BF16 divergence, downstream task performance was not systematically degraded.

    Hugging Face Daily Papersarxiv.org1 minpaper

  3. Dynamically Scaled Activation Steering

    Dynamically Scaled Activation Steering (DSAS) is a method‑agnostic framework that learns per‑token, per‑layer scaling factors to turn existing activation‑steering interventions on only when a model is likely to produce undesired output (e.g., toxic text). The scaling can be optimized jointly with any steering function, improves the toxicity‑utility trade‑off on language models, transfers to text‑…

    Apple Machine Learning Researchapple.com1 minpaper

  6. Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem

    The authors cast transformer block removal as a constrained binary optimization problem equivalent to an Ising glass, using a Hessian‑derived energy as a proxy for downstream quality. Solving the resulting QUBO with classical or quantum‑inspired solvers yields up to 23 MMLU points improvement over prior block‑removal baselines at 50 % depth compression.

    Hugging Facehuggingface.co8 min