A Practical Introduction to Control, Numerics and Machine Learning
Enrique Zuazua and Daniël Veldman
In this course we shall present a general introduction and some recent developments on the interface between Control, Numerics, and Machine Learning (Supervised Learning and Universal Approximation).
The first day is devoted to classical Control Theory with an emphasis on Pontryagin’s maximum principle, which is of great theoretical interest but is also of crucial importance for the efficient numerical computation of optimal controls. This importance is illustrated by an industrial application from precision engineering, in which a thermomechanical model needs to be controlled with nanometer accuracy.
On the second day, we approach Supervised Learning from a dynamical systems perspective. We show that the training of Deep Residual Neural Networks (ResNets) and their continuous time counterparts (Neural ODEs) have close connections with classical (optimal) control theory and that these similarities are also visible in the numerics. It is demonstrated how these ideas lead to efficient numerical algorithms for Supervised Learning with deep ResNets (with an emphasis on classification problems). We also discuss the simultaneous or ensemble controllability property of ResNets, which shows that Deep ResNets possess the Universal Approximation property, a property that linear control systems do not have.
The third day is devoted to more advanced topics such as the turnpike property in deep learning, momentum ResNets and Neural Transport Equations, and stochastic algorithms like Stochastic Gradient Descent and the Random Batch Method.
Every day is concluded by a practical session in which the theory discussed that day is applied and implemented in MATLAB. A basic working knowledge of MATLAB (or similar software) is therefore recommended.
Organization
The summer school will take place from September 4 to 6, 2022.
- Registration will be on Sep. 04, 2022 from 8:30 to 9:00 am in front of the lecture hall.
- Lectures will start at 9:00 am and will end at approximately 3:30 pm.
- Coffee breaks from approximately 10:30 to 11:00 am in the foyer of the lecture hall.
- Lunch break from approximately 12:30 to 2:00 pm. Except on Sep. 04 lunch will be served in the Mensa II (Leibnizstraße 14, 24118 Kiel, see map below). The daily menu can be found here.
Instructions to reach Mensa II from the lecture hall can be found here
- https://www.openstreetmap.org/?mlat=54.34578&mlon=10.10889#map=18/54.34578/10.10889
- https://www.google.com/maps/dir/Olshausenstra%C3%9Fe+75,+Kiel/Mensa+II,+Leibnizstra%C3%9Fe,+Kiel/@54.3455591,10.1084942,18.54z/data=!3m1!5s0x47b25609df44f9db:0x3308df368a610d45!4m14!4m13!1m5!1m1!1s0x47b255e158025175:0xf76f56bf0377183c!2m2!1d10.10575!2d54.3454!1m5!1m1!1s0x47b25609ea985e6f:0x7e1d9dc118d6904f!2m2!1d10.1132262!2d54.3463985!3e2?hl=en
Location
The Summer School will take place at
Lecture Hall OS75 - Hans-Heinrich-Driftmann-Hörsaal
Olshausenstraße 75
24118 Kiel
The lecture hall can be reached by public transport with different bus lines. The closed bus stops are Kiel Bremerskamp and Leibnizstraße.
Schedule
| Day 1 (Sep. 04) | Day 2 (Sep. 5) | Day 3 (Sep. 6) |
| Registration from 8:30 to 9:00 am | - | - |
| Start: 9:00 am | Start: 9:00 am | Start: 9:00 am |
| Introduction to Control Theory | Introduction to Machine Learning and Neural ODEs | Advanced topic 1: E.g. the Turnpike Property in Deep Learning |
| Optimal Control & Pontryagin’s Maximum Principle | The Universal Approximation Property in Neural ODEs | Advanced topic 2: E.g. Momentum ResNets and Neural Transport Equations |
| Time-discretization of Optimal Control Problems | Deep learning as an Optimal Control Problem and Backpropagation | Stochastic Gradient Descent & Random Batch Methods |
| MATLAB exercise: Optimal control for a 2-D heat equation | MATLAB exercise: A deterministic algorithm for the training of DNNs | MATLAB exercise: Stochastic algorithms for the training of DNNs |
| End (approx.): 3:30 pm | End (approx.): 3:30 pm | End (approx.): 3:30 pm |
Speakers
Enrique Zuazua Iriondo (Eibar, Basque Country – Spain, 1961) holds a Chair of Dynamics, Control and Numerics – Alexander von Humboldt Professorship at FAU- Friedrich–Alexander University, Erlangen–Nürnberg (Germany). He also leads the research project “DyCon: Dynamic Control”, funded by the ERC – European Research Council at the Department of Mathematics, at UAM – Autonomous University of Madrid and Deusto Foundation, University of Deusto – Bilbao (Basque Country, Spain), where he holds secondary appointments as Professor of Applied Mathematics (UAM) and Director of CCM – Chair of Computational Mathematics (Deusto).
His research in the area of Applied Mathematics covers topics in Partial Differential Equations, Systems Control, Numerical Analysis and Machine Learning, and led to fruitful collaborations in different industrial sectors such as the optimal shape design in aeronautics, the management of electrical and water distribution networks and the design of recommendation systems. His research had a high impact (h-index 44) and he has mentored a significant number of postdoctoral researchers and coached a wide network of Science managers.
He holds a degree in Mathematics from the University of the Basque Country, and a dual PhD degree from the same university (1987) and the Université Pierre et Marie Curie, Paris (1988). In 1990 he became Professor of Applied Mathematics at the Complutense University of Madrid, to later move to UAM in 2001. He has been awarded the Euskadi (Basque Country) Prize for Science and Technology 2006 and the Spanish National Julio Rey Pastor Prize 2007 in Mathematics and Information and Communication Technology, the Advanced Grants NUMERIWAVES in 2010 and DyCon in 2016 of the European Research Council (ERC) and the SIAM W.T. and Idalia Reid Prize 2022. He is an Honorary member of the of Academia Europaea and Jakiunde, the Basque Academy of Sciences, Letters and Humanities, Doctor Honoris Causa from the Université de Lorraine in France and Ambassador of the Friedrisch-Alexandre University in Erlangen-Nurenberg, Germany. He was an invited speaker at ICM2006 in the section on Control and Optimization.
From 1999-2002 he was the first Scientific Manager of the Panel for Mathematics within the Spanish National Research Plan and the Founding Scientific Director of the BCAM – Basque Center for Applied Mathematics from 2008-2012. He is also a member of the Scientific Council of a number of international research institutions such as the INSMI-CNRS and CERFACS in France and member of the Editorial Board in some of the leading journals in Applied Mathematics and Control Theory.
Email: enrique.zuazua@fau.de
Daniël Veldman was born on June 9, 1990 in Nijmegen, the Netherlands. After finishing his secondary eduction at the Karel de Grote College in Nijmegen in 2008, he started studying Mathematics at Utrecht University (UU) in Utrecht, the Netherlands, which he combined with Mechanical Engineering at Eindhoven University of Technology (TU/e) in Eindhoven, the Netherlands, since 2009. He received his Bachelor of Science and Master of Science degrees in Mathematics from the UU in 2012 and 2016, respectively, and his Bachelor of Science and Master of Science degrees in Mechanical Engineering from the TU/e in 2012 and 2015, respectively. He received his PhD degree on September 11, 2020 from Eindhoven University of Technology.
Since October 2020, Daniël is working as a postdoctoral researcher at the Chair for Dynamics, Control, and Numerics at the Department of Data Science at the Friedrich-Alexander Universität Erlangen-Nuremberg (FAU). Under supervision of Enrique Zuazua, he is working on problems at the interface of control theory, numerical analysis, and machine learning.
Email: daniel.wm.veldman@fau.de