CMSA Member Seminar: Optimal learning protocols via statistical physics and control theory

Behind the impressive performance of modern machine learning lies a toolkit of training tricks, from tuning learning rates to curating training data. These heuristics are powerful but hard to interpret and possibly suboptimal, leaving open the challenge of finding general principles for protocol design. In this talk, I will present a framework that combines tools from statistical physics and control theory to identify optimal training strategies in simple yet insightful neural network models. In the high-dimensional limit, the training dynamics can be reduced to closed-form ordinary differential equations for a small set of order parameters that track learning. This reduction allows us to pose the design of training protocols as an optimal control problem directly on the order-parameter dynamics, with the objective of minimizing the generalization error. This formulation encompasses a variety of learning scenarios and yields principled training strategies that clarify, and in some cases improve upon, standard heuristic practices.