1
Sample Count Is Not Enough: Candidate-Generation Strategy Shapes the Energy and Performance of LLM Test-Time Scaling
Increasing the number of LLM candidates (N) improves reasoning accuracy, but the way those candidates are generated (batch size vs sequential calls) dramatically affects latency, GPU‑hours, and energy. On A100 GPUs, eight serial 1‑candidate calls consume ~5× more energy and take ~6× longer than a single 8‑candidate batched call, even though total candidate count is identical. The authors recommen…
Hugging Face Daily Papersarxiv.org2 minpaper
