proomt

Search

Search posts, papers, and topics

All posts

Hugging Face Daily PapersXiaoqiang Wang, Mengyang Xiong, Jun Dai1 min readpaperadvanced

Circuit Hypernetworks for Quantum-Augmented Diffusion Language Models

Read on arxiv.org

Summary

The paper introduces HyperQ, a token‑conditioned quantum residual branch added to a frozen masked‑diffusion language model. By using a hypernetwork to generate circuit parameters and an exact linear‑time classical expectation formula, they train circuits up to 64 qubits and achieve modest benchmark improvements over the base model with far fewer fine‑tuning examples.

  • HyperQ inserts a quantum residual module per transformer block that reads token hidden states, runs a parameterised circuit, and adds measured expectations back via a residual connection.
  • A lightweight hypernetwork emits token‑specific rotation angles, coupling strengths, and measurement axes for a shared sparse circuit, keeping the quantum part small and trainable.
  • Expectation values of the circuits have a closed‑form classical expression whose cost scales linearly with qubit count, enabling training of 16‑64 qubit circuits within a 1.1 B‑parameter model.
  • Across downstream tasks, increasing circuit width raises average scores from 47.65 to 54.30, beating the frozen backbone by 4.71 points and a low‑rank adaptation by 3.67 points.

Engineers and researchers interested in practical quantum‑augmented language models and low‑cost adaptation techniques should see how token‑conditioned circuits can be integrated efficiently.

6/10
llmdiffusiontransformersquantum computinghypernetworks

Related reading

  1. A theoretical separation between quantum computers & LLMs

    The IBM research blog explains two new theoretical results that prove shallow constant‑depth quantum circuits can outperform decoder‑only transformers on a functional task (iterated index) and diffusion language models on a sampling task (parity‑sampling). The proofs give asymptotic separations but are not yet practical.

    IBM Researchibm.com6 min

  2. Flash-dLLM: IO-Aware KV Caching and Parallel Decoding for Fast, Memory-Efficient Diffusion LLMs

    Flash-dLLM is a training-free framework that accelerates Diffusion LLM inference by addressing GPU memory I/O bottlenecks with an I/O-aware KV-cache kernel. It also introduces a KV-cache-driven draft-and-verify decoding strategy, achieving significant speedups (up to 11x) over prior methods.

    Hugging Face Daily Papersarxiv.org1 minpaper

  4. From error mitigation to fault-tolerant quantum computing

    IBM’s blog outlines a practical continuum from quantum error mitigation to fault‑tolerant error correction, highlighting hybrid spacetime codes that achieve ~10× effective error‑rate reduction (64 logical qubits on 76 physical qubits) and a 63× sampling‑overhead cut versus pure mitigation. It describes how post‑selected QEC combined with probabilistic error cancellation, dynamic circuits, and a n…

    IBM Researchibm.com10 min

  5. Register Tokens for Bounded-State Reasoning in Diffusion Language Models

    Register tokens are fixed‑position embeddings that store a compact hidden state across diffusion‑based language model generation chunks, enabling bounded‑state reasoning without retaining all prior text. Post‑training on LLaDA and Dream shows up to +8.5 math and +19.5 code benchmark points versus plain text carry, and RL fine‑tuning further improves long‑horizon tasks.

    Hugging Face Daily Papersarxiv.org1 minpaper

  6. SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization

    This paper introduces SpectralShift, a spectral reparameterization method for extending the context window of Gated DeltaNet (GDN) linear attention models. It reconfigures the decay spectrum by enhancing slow propagation and preserving fast-decaying modes, consistently improving long-context capabilities during continual pretraining.

    Hugging Face Daily Papersarxiv.org1 minpaper