ml theory

Exploding variance of means of exponentials: least-squares to the rescuefrancisbach.com

This article addresses the exploding variance problem when estimating log-sum-exp functions, common in machine learning, using empirical averages. It proposes a novel approach that re-frames the log-sum-exp as an integral over weighted chi-square divergences, allowing for stable, closed-form least-squares estimators.