2025-2026
Fall 2025 Seminar Series
Do Large Language Models (Really) Need Statistical Foundations?
Friday, October 17, 2025
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Weijie Su, Associate Professor, Wharton Statistics and Data Science Department, University of Pennsylvania
Abstract: In this talk, we advocate for developing statistical foundations for large language models (LLMs). We begin by examining two key characteristics that necessitate statistical perspectives for LLMs: (1) the probabilistic, autoregressive nature of next-token prediction, and (2) the inherent complexity and black box nature of Transformer architectures. To demonstrate how statistical insights can advance LLM development and applications, we present two examples. First, we demonstrate statistical inconsistencies and biases arising from the current approach to aligning LLMs with human preference. We propose a regularization term for aligning LLMs that is both necessary and sufficient to ensure consistent alignment. Second, we introduce a novel statistical framework for analyzing the efficacy of watermarking schemes, with a focus on a watermarking scheme developed by OpenAI for which we derive optimal detection rules that outperform existing ones. Time permitting, we will explore how statistical principles can inform rigorous evaluation for LLMs. Collectively, these findings demonstrate how statistical insights can effectively address several pressing challenges emerging from LLMs. This talk is based on arXiv:2404.01245, 2405.16455, 2503.10990, 2505.19145, and 2506.12350.
Phase Transitions in Estimation with Low-degree Polynomials
Friday, October 24, 2025
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Youngtak Sohn, Assistant Professor, Applied Mathematics, Brown University
Abstract: High-dimensional planted problems, such as finding a hidden dense subgraph within a random graph, often exhibit a gap between statistical and computational feasibility. While recovering the hidden structure may be statistically possible, it is conjectured to be computationally intractable in certain parameter regimes. A powerful approach to understanding this hardness involves proving lower bounds on the efficacy of low-degree polynomial algorithms. In this talk, I will introduce the low-degree polynomial framework and explain how it captures key features of algorithmic hardness. I will then discuss recent progress on understanding computational barriers in community detection under the Stochastic Block Model with many communities. This is joint work with Byron Chin, Elchanan Mossel, and Alex Wein.
Trees Against Gerrymandering
Friday, October 31, 2025
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Moon Duchin, Professor of Computer Science and Data Science, University of Chicago
Abstract: Motivated by the study of political redistricting, many mathematicians have gotten interested in sampling algorithms for graph partitions. (In this case the graph is a contact network of geographic units in a state.) There has been quite a lot of recent progress developing spanning-tree methods to do the sampling, and I'll survey some of what is and is not known. Bonus: I'll show you how this is being used in the current redistricting court cases in Texas!
Statistical Inference for Temporal Difference Learning with Linear Function Approximation
Friday, November 7, 2024
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Alessandro Rinaldo, Professor, Department of Statistics and Data Sciences, The University of Texas at Austin
Abstract: Policy evaluation is a fundamental task in Reinforcement Learning (RL), with applications in numerous fields, such as clinical trials, mobile health, robotics, and autonomous driving. Temporal Difference (TD) learning and its variants are arguably the most widely used algorithms for policy evaluation with linear approximation. Despite the popularity and practical importance of TD estimators of the parameters of the best linear approximation to the value function, theories and methods for formal statistical inference with finite sample validity in high dimensions remain limited. Consequently, RL practitioners often lack essential statistical tools to guide their decision-making. To address this gap, we develop efficient inference procedures for TD learning-based estimators under linear function approximation in on-policy settings. We obtain improved consistency rates and derive novel high-dimensional Berry-Esseen bounds for the TD estimator under independent samples and Markovian trajectories. Additionally, we propose an online algorithm to construct non-asymptotic confidence intervals for the target parameters.
Joint work with Weichen Wu (Voleon) and Yuting Wei (UPenn).
How do Neural Networks Learn Features from Data?
Friday, November 14, 2025
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Adityanarayanan Radhakrishnan, Assistant Professor of Mathematics, Massachusetts Institute of Technology
Abstract: The ability of neural networks to learn patterns from data, or features, has been central to their success. In this talk, I will present a unifying mechanism that characterizes feature learning across neural network architectures. Namely, features learned by neural networks are captured by a statistical operator known as the average gradient outer product (AGOP). More generally, the AGOP enables feature learning in machine learning models that have no built-in feature learning mechanism (e.g., kernel methods). I will present two applications of this line of work. First, I will show how AGOP can be used to steer LLMs and vision-language models, guiding them towards specified concepts and shedding light on vulnerabilities in these models. I will then discuss how AGOP connects feature learning with independence testing and how we used AGOP to develop a scalable, nonlinear measure of dependence known as the InterDependence Score (IDS). I will conclude with an application of IDS to million-scale text and genomics datasets, where we use it to identify subpopulations of interest.
Winter 2026 Seminar Series
Towards the Last Mile of Artificial General Intelligence: Open-World Long-Tailed Learning in Theory and Practice
Friday, January 23, 2026
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Dawei Zhou, Assistant Professor, Department of Computer Science, Virginia Tech
Abstract: Artificial General Intelligence (AGI) represents the next generation of AI that can match or exceed human intelligence across a wide spectrum of tasks. Despite remarkable advances, today’s AI systems succeed mainly in data-rich, well-structured settings—identifying common objects, summarizing routine content, or responding to typical queries. They struggle precisely where intelligence matters most—rare, high-stakes, and context-dependent scenarios such as scientific discovery, open-world cybersecurity, and rare disease diagnosis. We argue that this shortfall defines the Last Mile Problem on the path to AGI, which we frame as Open-World Long-Tailed Learning (OpenLT): how can we enable AI systems to reason, adapt, and generalize across the underrepresented, evolving, and open-ended domains? In this talk, I will discuss our group’s recent work on 1) OpenLT Characterization – How can we systematically characterize and uncover novel, complex patterns in open-world data?, 2) OpenLT Adaptation – How can AI models be effectively adapted to open and dynamic environments?, and 3) OpenLT Application and Deployment - hinging on the application of scientific hypothesis generation for 3D metamaterial design to discuss our proposed techniques and theoretical results for open-world long-tailed learning. Finally, I will close with thoughts on how addressing the Last Mile Problem can shape the next decade of AGI research and move us closer to systems that truly understand and operate in the open world.
Graph Neural Network Meets Random Geometric Graph
Friday, February 13, 2026
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Suqi Liu, Assistant Professor, Department of Statistics, University of California, Riverside
Abstract: Graph neural networks (GNNs) have emerged as a powerful framework for learning from graph-structured data, yet their theoretical understanding—particularly regarding the behavior of different architectural choices across various graph-based tasks—remains limited. In parallel, random geometric graphs (RGGs) provide a well-defined probabilistic model that captures the interplay between geometry and connectivity in complex networks. In this talk, I will discuss several efforts I have undertaken to bridge these two perspectives by studying GNNs through the lens of RGGs. In the first part, I will focus on the classic graph matching problem and show that, by leveraging a specific GNN, perfect recovery can be achieved even in high-noise regimes. In the second part, I will briefly highlight recent work demonstrating the provable benefits of graph attention networks (GATs) for a node regression task. This talk is based on joint work with Morgane Austern, Kenny Gu, and Somak Laha.
Deep Survival Learning for Kidney Transplantation: Knowledge Distillation and Data Integration
Friday, February 20, 2026
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Kevin He, Associate Professor of Biostatistics and Associate Director of the Kidney Epidemiology and Cost Center (KECC), University of Michigan
Abstract: Prognostic prediction using survival analysis faces challenges due to complex relationships between risk factors and time-to-event outcomes. Deep learning methods have shown promise in addressing these challenges, but their effectiveness often relies on large datasets. However, when applied to moderate- or small-sized datasets, deep models frequently encounter limitations such as insufficient training data, overfitting, and difficulty in hyperparameter optimization. To mitigate these issues and enhance prognostic performance, this talk presents a flexible deep learning framework that integrates external risk scores with internal time-to-event data through a generalized Kullback–Leibler divergence regularization term. Applied to the national kidney transplant data, the proposed method demonstrates improved prediction of short-term mortality and graft failure following kidney transplantation by distilling and transferring prior knowledge from pre-policy-change teacher models to newly arrived post-policy-change cohorts.
What functions does XGBoost learn?
Friday, February 27, 2026
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Aditya Guntuboyina, Associate Professor, Department of Statistics, University of California, Berkeley
Abstract: We develop a theoretical framework that explains what kinds of functions XGBoost is able to learn. We introduce an infinite-dimensional function class that extends ensembles of shallow decision trees, along with a natural measure of complexity that generalizes the regularization penalty built into XGBoost. We show that this complexity measure aligns with classical notions of variation—in one dimension it corresponds to total variation, and in higher dimensions it is closely tied to a well-known concept called Hardy–Krause variation. We prove that the best least-squares estimator within this class can always be represented using a finite number of trees, and that it achieves a nearly optimal statistical rate of convergence, avoiding the usual curse of dimensionality. Our work provides the first rigorous description of the function space that underlies XGBoost, clarifies its relationship to classical ideas in nonparametric estimation, and highlights an open question: does the actual XGBoost algorithm itself achieve these optimal guarantees? This is joint work with Dohyeong Ki at UC Berkeley.
Spring 2026 Seminar Series
Feature Learning in Kernel Machines and Applications to Monitoring and Steering LLMs
Friday, April 17, 2026
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Mikhail Belkin, HDSI Endowed Chair Professor in AI, Halicioglu Data Science Institute, University of California San Diego
Abstract: Classical kernel machines are a powerful and theoretically grounded method for data analysis. However, they are not adaptive to low-dimensional "features" in the data.
In this talk I will discuss feature learning introducing Recursive Feature Machines—a powerful method designed for extracting relevant features from tabular data. I will discuss some of its interesting properties and, in particular, will show how this technique enables us to detect and precisely guide LLM behaviors toward almost any desired concept by manipulating a single fixed vector in the LLM activation space.
When do Spectral Gradient Updates Help in Deep Learning?
Friday, May 1, 2026
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Dmitriy Drusvyatskiy, Professor and HDSI Faculty Fellow, Halıcıoğlu Data Science Institute (HDSI), University of California San Diego
Abstract: Spectral gradient methods, such as the recently proposed Muon optimizer, are a promising alternative to standard gradient descent for training deep neural networks and transformers. Yet, it remains unclear in which regimes these spectral methods are expected to perform better. In this talk, I will present a simple condition that predicts when a spectral update yields a larger decrease in the loss than a standard gradient step. Informally, this criterion holds when, on the one hand, the gradient of the loss with respect to each parameter block has a nearly uniform spectrum—measured by its nuclear-to-Frobenius ratio—while, on the other hand, the incoming activation matrix has low stable rank. It is this mismatch in the spectral behavior of the gradient and the propagated data that underlies the advantage of spectral updates. Reassuringly, this condition naturally arises in a variety of settings, including random feature models, neural networks, and transformer architectures. I will conclude by showing that these predictions align with empirical results in synthetic regression problems and in small-scale language model training.
Words matter: Multimodal Suicide Risk Prediction from Veterans Health Administration Clinical Notes
Friday, May 8, 2026
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Jiang Gui, Associate Professor, Biomedical Data Science, Dartmouth College
Abstract: In this talk, we demonstrate that integrating unstructured clinical narratives with structured electronic health record (EHR) data enhances suicide risk prediction for U.S. Veterans, outperforming models that rely on structured data alone. By analyzing a retrospective matched case-control cohort of 4,584 Veterans who died by suicide and 22,657 controls, we compared traditional count-based text features against pretrained contextual large language model (LLM) embeddings, such as Clinical Longformer and BioClinicalBERT. We found that while Adaptive Mixture Categorization (AMC) improves the utility of skewed linguistic data, contextual LLM embeddings consistently provide comparable or superior predictive power, particularly within low- and moderate-risk tiers where structured indicators may be less obvious. Our multimodal approach, which integrated 66 structured patient characteristics with text features, yielded substantial performance gains, increasing AUROC by approximately 0.07–0.11 across various risk tiers and time windows. Furthermore, our temporal analysis revealed that while long-term data (270 days) is most informative for low-risk patients, short-term windows (<30 days) are critical for high-risk individuals. Using SHAP-based interpretability and topic modeling, we identified clinically coherent themes that shift semantically as risk increases, providing a context-aware framework for improving suicide prevention efforts within the Veterans Health Administration.
Data-Centric Learning: Aligning Data and Model Knowledge for Better AI
Friday, May 15, 2026
Time: 11:00 a.m. to 12:00 p.m. central time
Location: Ruan Conference Room – lower level (Chambers Hall 600 Foster Street)
Speaker: Yanjie Fu, Associate Professor, School of Computing and Augmented Intelligence, Arizona State University
Abstract: Recent progress in AI has been driven largely by scaling models and compute. Yet in many real-world and scientific settings, AI failures are still rooted less in model architecture than in the data itself: missing or incomplete observations, noisy labels, distribution shift, imbalance, poor feature geometry, and weak coverage of the underlying domain. This talk argues for a shift from a model-centric view of AI to a data-centric learning perspective, where the central goal is not only to train better models, but to reshape better data for learning. I will present a unifying view of data-centric learning through the lens of data-model knowledge alignment: data serves as the knowledge base, models learn knowledge from data, and poor alignment between the two leads to poor generalization, shortcut learning, instability, and low trust. I will introduce key directions in this space, including data curation, relabeling, synthetic data generation, feature selection, feature transformation, and data reprogramming. I will also highlight our recent work on AI4Data-RL, AI4Data-GenAi, AI4Data-LLM&Agents. Overall, the talk will discuss how data-centric learning opens a new path toward more robust and trustworthy AI systems.