Score Matching

I recently posted a new preprint titled Energy-Tweedie: Score meets Score, Energy meets Energy on ArXiv.

We first extend the classic Tweedie’s formula to a wider range of distributions, then derive a novel identity that links the Stein score under a generalized Gaussian (energy model) noise to the gradient of a matched Energy Score (a scoring rule), connecting two seemingly disparate approaches to generative modeling. Among other things, this enables energy-score-based denoising models to be plugged into standard diffusion pipelines, but with heavy‑tailed noise.

For the Distributional Principal Autoencoder (DPA), we prove an exact identity linking the data score to the geometry of the encoder level sets, and show that any latent coordinates beyond the data manifold become completely uninformative. This means that the DPA learns nonlinear manifolds shaped locally by the data density, with a clear, testable dimensionality criterion — conditional independence, furthering the PCA analogy.