Energy-Tweedie score fields for the Eight-Gaussians data noised with generalized Gaussian noise. Each figure foregrounds the effect of a single noise parameter.Abstract
Denoising and score estimation are classically linked through Tweedie’s formula, which relates the posterior mean under Gaussian noise to the Stein score of the noisy marginal. In this work, we extend this perspective beyond Gaussian noise to a broad class of Gibbs (energy-based) noise distributions, with the generalized Gaussian family as the running example.
We derive the Energy-Tweedie identity: when the denoising posterior is viewed through the lens of scoring rules, the path derivative of a kernel scoring rule defined by the noise potential recovers the Stein score of the noisy marginal. The rule’s propriety is determined by the noise potential alone. Thus, the familiar correspondence between Gaussian noise, posterior means, squared loss, and Tweedie’s formula is lifted to a distributional correspondence between Gibbs noise distributions, full posterior laws, kernel scoring rules, and the Energy-Tweedie identity, yielding one Tweedie-style relation for each noise potential.
Among its consequences, this identity gives a posterior-samples-to-score route to score estimation, yields a principled criterion for estimating unknown noise parameters, and enables diffusion-style sampling along user-chosen paths through the noise-parameter space, supplying the score-based perspective on recent generative methods trained with scoring rules.
Classical Tweedie’s formula links Gaussian corruption, squared-error denoising, the posterior mean, and the score of the noisy data. We present the Energy–Tweedie identity, which generalizes this correspondence from a mean-based relation to a distributional one and holds for any Gibbs (“energy-based”) noise distribution. Each noise distribution induces a kernel scoring rule, whose path derivative evaluated at the denoising posterior gives the noisy-data score. In the Gaussian noise case, this reduces exactly to classical Tweedie’s formula.
The identity has three main consequences:
- The score can be estimated from samples from a denoising posterior model (i.e., a conditional generative model).
- All the parameters of the noise distribution (within the Gibbs family) can be recovered from corrupted data in a principled fashion.
- It provides a score-based perspective on diffusion approaches based on scoring rules; among other consequences, existing score-based samplers can thus be used to generate from such models, with the path through the (multidimensional) noise-parameter space a free design choice at sampling time - illustrated by the MNIST samples below.
