ITA | ENG
B030868 - Numerical Optimization with Applications to Machine Learning
Main information
Teaching Language
Course Content
Suggested readings
Learning Objectives
Prerequisites
Teaching Methods
Further information
Type of Assessment
Course program
Academic Year 2020-21
Coorte 2020 - Second Cycle Degree in MATHEMATICS
Course year
First year - Second Semester
Belonging Department
Mathematics and Computer Science "Ulisse Dini" (DIMAI)
Course Type
Single education field course
Scientific Area
MAT/08 - NUMERICAL ANALYSIS
Credits
9
Teaching Hours
72
Teaching Term
22/02/2021 ⇒ 11/06/2021
Attendance required
No
Type of Evaluation
Final Grade
Course Content
show
Course program
show
Lectureship
Teaching Language
Italian
Course Content
Numerical methods for unconstrained nonlinear programming. Stochastic gradient type methods for optimization problems in machine-learning.
Optimality conditions for constrained optimization. Methods for Linear Programming problems. Basic Python instructions and scipy.optimize module.
Applications to classification problems and neural-networks training.
Matlab Optimization Toolbox.
Optimality conditions for constrained optimization. Methods for Linear Programming problems. Basic Python instructions and scipy.optimize module.
Applications to classification problems and neural-networks training.
Matlab Optimization Toolbox.
Suggested readings (Search our library's catalogue)
J. Nocedal, S.J. Wright, "Numerical Optimization", 2nd ed., 2006
I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, The MIT Press, 2016.
G. Strang, Linear algebra and learning from data,
Wellesley-Cambridge Press, 2019.
I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, The MIT Press, 2016.
G. Strang, Linear algebra and learning from data,
Wellesley-Cambridge Press, 2019.
Learning Objectives
Knowledge of optimality theory for linear programming and constrained nonlinear programming.
Knowledge of the main numerical optimization methods and their theoretical background with focus on:
unconstrained optimization problems, problems arising in machine-learning and linear programming problems.
Knowledge of the main numerical optimization methods and their theoretical background with focus on:
unconstrained optimization problems, problems arising in machine-learning and linear programming problems.
Prerequisites
Courses recommended: first level courses of Mathematical Analysis, Numerical Analysis and Probability.
Teaching Methods
Frontal lessons and experience in laboratory.
Further information
Frequency of lab lessons: Recommended
Teaching Tools: additional notes, slides, UniFi E-Learning: http://e-l.unifi.it
Teaching Tools: additional notes, slides, UniFi E-Learning: http://e-l.unifi.it
Type of Assessment
Oral exam about lecture’s topics and laboratory activities.
Course program
Unconstrained nonlinear programming: optimality conditions. Gradient method and Conjugate Gradient method for objective quadratic functions. Gradient method, Newton method and quasi-Newton methods for nonlinear objective functions.
Gauss-Newton method for nonlinear least-squares problems.
Line-search globalization techniques.
Optimization problems in machine-learning
(classification problems and neural networks training).
Linear Programming: models formulation with examples from optimal resource allocation, blending problems.
Introduction to Linear programming: basic feasible points, feasible polytope, Optimality and Duality Theory, Simplex method.
Primal Dual Interior Point methods: Introduction, central path, path-following methods, convergence theory.
Constrained Nonlinear Optimization: feasible directions, derivation of first order and second order conditions.
Introduction to Python language: basic instructions, functions, modules. Scipy.optimize module and applications to classification problems. Numerical solution of problems in neural-networks training via Keras.
Matlab Optimization Toolbox for linear programming.
Gauss-Newton method for nonlinear least-squares problems.
Line-search globalization techniques.
Optimization problems in machine-learning
(classification problems and neural networks training).
Linear Programming: models formulation with examples from optimal resource allocation, blending problems.
Introduction to Linear programming: basic feasible points, feasible polytope, Optimality and Duality Theory, Simplex method.
Primal Dual Interior Point methods: Introduction, central path, path-following methods, convergence theory.
Constrained Nonlinear Optimization: feasible directions, derivation of first order and second order conditions.
Introduction to Python language: basic instructions, functions, modules. Scipy.optimize module and applications to classification problems. Numerical solution of problems in neural-networks training via Keras.
Matlab Optimization Toolbox for linear programming.