Some of the research directions I have pursued:
Scoring rules are classical tools for evaluating probabilistic forecasts: for example, if you output a probability of rain tomorrow, a scoring rule can be used as a scalar measure of the quality of your probabilistic (i.e., distributional) predictions over time. The standard reference is Gneiting and Raftery (2007).
More recently, scoring rules have increasingly been used to train generative models, which can be viewed as outputting distributions. My research examines the structural properties of such models:
Distributional Autoencoders Know the Score: a nonlinear-PCA analogue
An autoencoder based on a scoring rule - the Distributional Principal Autoencoder (Shen & Meinshausen ‘24 ) - is proven to (1) define the encoding exactly in terms of the score function of the data distribution, and (2) permit the recovery, in principle, of the exact data manifold dimension by making the “extra” dimensions of the encoding uninformative.
Energy-Tweedie: Score meets Score, Energy meets Energy: Tweedie’s formula as a special case of a broader identity.
The Energy Score (a specific scoring rule) plays a crucial role in a new identity - the Energy-Tweedie identity - that generalizes the classic Tweedie’s formula, which is then shown to be a special case for a particular selection of parameters. This has several implications; for generative models specifically, it provides the score-based perspective on diffusion-like approaches using scoring rules.
CausalDS: Benchmarking Causal Reasoning in Data-Science Agents: a causal benchmark generator.
CausalDS is an end-to-end synthetic benchmark (generator) for jointly evaluating an LLM agent’s symbolic causal reasoning, data-science, uncertainty quantification, and coding/tool use skills, as well as whether it knows to abstain from unanswerable questions. The fully synthetic structure is crucial to disambiguating what we would consider “real causal reasoning” from repeating associations present in existing data, which is often termed the “causal parrot” failure mode.
Through Project CETI, which aims to further our understanding of sperm whale communication, I have worked on:
Approaching an unknown communication system by latent space exploration and causal inference: using generative models to identify potential carriers of meaning.
We introduce a method to interpret representations learned by GANs, which helps us identify several potential carriers of meaning in sperm whale communication, including specific acoustic properties; acoustic properties had previously not been considered meaningful.
Vowel- and Diphthong-Like Spectral Patterns in Sperm Whale Codas: from an AI-derived hypothesis to a concrete discovery.
Following these clues, we identify spectral patterns in sperm whale codas that are reminiscent of components of human speech. The patterns recur across whales, and appear to be actively controlled; this makes them promising candidates for carriers of meaning in sperm whale communication.
The work has received substantial media coverage.