Summer School

A Practical Introduction to Control, Numerics and Machine Learning

Enrique Zuazua and Daniël Veldman

In this course we shall present a general introduction and some recent developments on the interface between Control, Numerics, and Machine Learning (Supervised Learning and Universal Approximation).

The first day is devoted to classical Control Theory with an emphasis on Pontryagin’s maximum principle, which is of great theoretical interest but is also of crucial importance for the efficient numerical computation of optimal controls. This importance is illustrated by an industrial application from precision engineering, in which a thermomechanical model needs to be controlled with nanometer accuracy.

On the second day, we approach Supervised Learning from a dynamical systems perspective. We show that the training of Deep Residual Neural Networks (ResNets) and their continuous time counterparts (Neural ODEs) have close connections with classical (optimal) control theory and that these similarities are also visible in the numerics. It is demonstrated how these ideas lead to efficient numerical algorithms for Supervised Learning with deep ResNets (with an emphasis on classification problems). We also discuss the simultaneous or ensemble controllability property of ResNets, which shows that Deep ResNets possess the Universal Approximation property, a property that linear control systems do not have.

The third day is devoted to more advanced topics such as the turnpike property in deep learning, momentum ResNets and Neural Transport Equations, and stochastic algorithms like Stochastic Gradient Descent and the Random Batch Method.

Every day is concluded by a practical session in which the theory discussed that day is applied and implemented in MATLAB. A basic working knowledge of MATLAB (or similar software) is therefore recommended.

Preliminary Schedule

The summer school will take place from September 4 to 6, 2022.

Day 1 (Sep. 04)Day 2 (Sep. 5)Day 3 (Sep. 6)
Introduction to Control TheoryIntroduction to Machine Learning and Neural ODEsAdvanced topic 1: E.g. the Turnpike Property in Deep Learning
Optimal Control & Pontryagin’s Maximum PrincipleThe Universal Approximation Property in Neural ODEsAdvanced topic 2: E.g. Momentum ResNets and Neural Transport Equations
Time-discretization of Optimal Control ProblemsDeep learning as an Optimal Control Problem and BackpropagationStochastic Gradient Descent & Random Batch Methods
MATLAB exercise: Optimal control for a 2-D heat equationMATLAB exercise: A deterministic algorithm for the training of DNNsMATLAB exercise: Stochastic algorithms for the training of DNNs

Speakers

Enrique Zuazua Iriondo (Eibar, Basque Country – Spain, 1961) holds a Chair of Dynamics, Control and Numerics – Alexander von Humboldt Professorship at FAU- Friedrich–Alexander University, Erlangen–Nürnberg (Germany). He also leads the research project “DyCon: Dynamic Control”, funded by the ERC – European Research Council at the Department of Mathematics, at UAM – Autonomous University of Madrid and Deusto Foundation, University of Deusto – Bilbao (Basque Country, Spain), where he holds secondary appointments as Professor of Applied Mathematics (UAM) and Director of CCM – Chair of Computational Mathematics (Deusto).

His research in the area of Applied Mathematics covers topics in Partial Differential Equations, Systems Control, Numerical Analysis and Machine Learning, and led to fruitful collaborations in different industrial sectors such as the optimal shape design in aeronautics, the management of electrical and water distribution networks and the design of recommendation systems. His research had a high impact (h-index 44) and he has mentored a significant number of postdoctoral researchers and coached a wide network of Science managers.

He holds a degree in Mathematics from the University of the Basque Country, and a dual PhD degree from the same university (1987) and the Université Pierre et Marie Curie, Paris (1988). In 1990 he became Professor of Applied Mathematics at the Complutense University of Madrid, to later move to UAM in 2001. He has been awarded the Euskadi (Basque Country) Prize for Science and Technology 2006 and the Spanish National Julio Rey Pastor Prize 2007 in Mathematics and Information and Communication Technology, the Advanced Grants NUMERIWAVES in 2010 and DyCon in 2016 of the European Research Council (ERC) and the SIAM W.T. and Idalia Reid Prize 2022. He is an Honorary member of the of Academia Europaea and Jakiunde, the Basque Academy of Sciences, Letters and Humanities, Doctor Honoris Causa from the Université de Lorraine in France and Ambassador of the Friedrisch-Alexandre University in Erlangen-Nurenberg, Germany. He was an invited speaker at ICM2006 in the section on Control and Optimization.

From 1999-2002 he was the first Scientific Manager of the Panel for Mathematics within the Spanish National Research Plan and the Founding Scientific Director of the BCAM – Basque Center for Applied Mathematics from 2008-2012. He is also a member of the Scientific Council of a number of international research institutions such as the INSMI-CNRS and  CERFACS in France and member of the Editorial Board in some of the leading journals in Applied Mathematics and Control Theory.

Email: enrique.zuazua@fau.de

Website: https://dcn.nat.fau.eu/enrique-zuazua/

Daniël Veldman was born on June 9, 1990 in Nijmegen, the Netherlands. After finishing his secondary eduction at the Karel de Grote College in Nijmegen in 2008, he started studying Mathematics at Utrecht University (UU) in Utrecht, the Netherlands, which he combined with Mechanical Engineering at Eindhoven University of Technology (TU/e) in Eindhoven, the Netherlands, since 2009. He received his Bachelor of Science and Master of Science degrees in Mathematics from the UU in 2012 and 2016, respectively, and his Bachelor of Science and Master of Science degrees in Mechanical Engineering from the TU/e in 2012 and 2015, respectively. He received his PhD degree on September 11, 2020 from Eindhoven University of Technology.

Since October 2020, Daniël is working as a postdoctoral researcher at the Chair for Dynamics, Control, and Numerics at the Department of Data Science at the Friedrich-Alexander Universität Erlangen-Nuremberg (FAU). Under supervision of Enrique Zuazua, he is working on problems at the interface of control theory, numerical analysis, and machine learning.

Email: daniel.wm.veldman@fau.de

Website: https://dcn.nat.fau.eu/daniel-veldman/