Some of the research directions I have pursued:
Scoring rules are classical tools for evaluating probabilistic forecasts: for example, if you output a probability of rain tomorrow, a scoring rule can be used as a scalar measure of the quality of your probabilistic (i.e., distributional) predictions over time. The standard reference is Gneiting and Raftery (2007).
More recently, scoring rules have increasingly been used to train generative models, which can be viewed as outputting distributions. My research examines the fundamental properties of such models:
Distributional Autoencoders Know the Score: a nonlinear-PCA analogue
An autoencoder based on a scoring rule - the Distributional Principal Autoencoder (Shen & Meinshausen ‘24 ) - is proven to:
(1) define the encoding exactly in terms of the score function of the data distribution, and
(2) discover and permit the recovery of (in principle) the exact data manifold dimension by making the “extra” dimensions of the encoding completely uninformative.
Energy-Tweedie: Score meets Score, Energy meets Energy: one Tweedie-style formula per energy-based noise distribution.
The Energy-Tweedie identity generalizes the classic Tweedie’s formula from Gaussian noise to a much wider family of “energy-based” (Gibbs) distributions: for each distribution, a kernel scoring rule is induced and the score of the noisy data is given by the path-derivative of the rule evaluated at the denoising posterior. This has several implications; for generative models specifically, it provides the score-based perspective on diffusion approaches based on scoring rules, opening the possibility to select a path through the (now-multidimensional) noise parameter space as a free design choice.
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents: a causal benchmark generator.
CausalDS is an end-to-end synthetic benchmark (generator) for jointly evaluating an LLM agent’s symbolic causal reasoning, data-science, uncertainty quantification, and coding/tool use skills, as well as whether it knows to abstain from unanswerable questions. The fully synthetic structure is crucial to disambiguating what we would consider “real causal reasoning” from repeating associations present in existing data, which is often termed the “causal parrot” failure mode.
Through Project CETI, which aims to further our understanding of sperm whale communication, I have worked on:
Approaching an unknown communication system by latent space exploration and causal inference: using generative models to identify potential carriers of meaning.
We introduce a method to interpret representations learned by GANs, which helps us identify several potential carriers of meaning in sperm whale communication, including specific acoustic properties; acoustic properties had previously not been considered meaningful.
Vowel- and Diphthong-Like Spectral Patterns in Sperm Whale Codas: AI-suggested hypothesis verified.
Following the clues given by Approaching …, we identify spectral patterns in sperm whale codas that are reminiscent of components of human speech. The patterns recur across whales, and appear to be actively controlled; this makes them promising candidates for carriers of meaning in sperm whale communication.
The work has received substantial media coverage.