Spring ’24 CSC@USC/CommNetS-MHI Seminar Series
 |
Ingvar Ziemann,
University of Pennsylvania
|
Abstract
In this talk I discuss some recent advances in supervised learning with dependent data. In particular, the emphasis of this talk is to provide an instance-optimal understanding of learning with dependent data for the square loss function. The approach I present yields rates that match and extend known asymptotics even without any realizability assumption. This stands in stark contrast to typical non-asymptotic results which exhibit variance proxies that are deflated multiplicatively by the mixing time of the underlying data-generating process. Indeed, our results instead scale additively with the mixing time and are thereby only affected by second order statistics in the leading term. The key to obtaining this scaling is the introduction of the notion of a weakly sub-Gaussian class, which allows us to invoke mixed tail generic chaining. This notion is general enough to nearly all cover smooth hypothesis classes and a wide range of parametric classes. As a motivating example, I will also discuss our recent work on multi-task learning. Even when the problem itself is realizable, the analysis of a natural “two-stage” estimator decomposes into two supervised learning problems: one which is realizable, and one which is not. In this setting, we demonstrate how our refined understanding of supervised learning with dependent data can be applied to extend and sharpen existing guarantees for iid multi-task learning.
Biosketch
Ingvar Ziemann is a postdoctoral researcher at the University of Pennsylvania. He received his PhD in November 2022 from the Division of Decision and Control Systems at The Royal Institute of Technology (KTH) under the supervision of Henrik Sandberg. His research is centered on using statistical and information theoretic tools to study learning-enabled control methods, with a current interest in studying how learning algorithms generalize in the context of dynamical systems. Prior to starting his Ph.D., he obtained two sets of Master's and Bachelor's degrees in Mathematics (SU/KTH) and in Economics and Finance (SSE). Ingvar is the recipient of a Swedish Research Council International Postdoc Grant, the 2022 IEEE Conference on Decision and Control Best Student Paper Award, and the 2017 Stockholm Mathematics Center Excellent Master Thesis Award.
Acknowledgement: seminar series is supported by the Ming Hsieh Institute and Quanser.