Generative Models

Energy-Tweedie: Score meets Score, Energy meets Energy has been accepted to NeurIPS 2026.

A 60-minute fireside chat on Distributional Autoencoders Know the Score, hosted by Ploutos.

A short announcement of the Energy-Tweedie preprint and its publication page.

The classical Tweedie’s formula connects the score under Gaussian noise to the posterior mean. We generalize this result to the Energy-Tweedie identity: for “energy-based” (Gibbs) noising distributions, the score is connected to a path-derivative of a kernel scoring rule induced by the noise distribution itself and evaluated at the denoising posterior. Among other things, this opens new routes to score and noise-parameter estimation, and provides the score-based perspective on diffusion approaches based on scoring rules - with the path through the (multidimensional) noise-parameter space a free design choice at sampling time.

For the Distributional Principal Autoencoder (DPA), we prove an exact identity linking the geometry of the learned encoding to the score of the data distribution, and show that any latent coordinates beyond the data manifold dimension become completely uninformative. This means that the DPA learns nonlinear manifolds shaped locally by the data density, with a clear, testable dimensionality criterion — conditional independence, giving it a natural nonlinear-PCA interpretation.

A talk at the Simons Institute on using deep generative models to study sperm whale communication.

We propose a novel methodology - Causal Disentanglement with Extreme Values (CDEV) - to identify representations learned by GANs. When trained on raw whale communication, it finds - for the first time - specific acoustic attributes that might serve as carriers of meaning.