Announcements             Syllabus             Schedule/Downloads

Schedule/Downloads (Fall 2021)

 orange-ball.gif The schedule is tentative and subject to change. (Last update: 08/08/24)

    - PLMS link

Week  Ch.# of 4th Ed. Topic 21-2 20-1 17-2 17-1
 Videos Onenotes  HWs Handouts  Videos pdfs HWs
1 　 Syllabus, Introduction 01 9/7 T 　 　 　 00  01 01 (pdf) 　
　 Examples. 02 9/9 Th 　 　 　 00  02 02 (pdf) 　
1, 2

Part I: Mathematical Review

 Math preliminary: Notation, real vector spaces, linear independence, matrices. HW#1 (Due on Fri. 9/24) 　 　 2-5  03 03 (pdf) HW1 (pdf)
2 3, 4, 5 Math preliminary: Inner product, norm. Eigenvalues and eigenvectors, quadratic forms, calculus of several variables, chain rule, Taylor series, gradient, level sets, directional derivative. 03 9/14 T  04 04 (pdf) 　
6 Definition of optimization problem and types of solutions. Quadratic problems. FONC. 04 9/16 Th HW#2 (Due 10/4) 　 　 6  05 05 (pdf)  
3 SONC and SOSC. Basic iterative algorithms: form, basic properties, line search. 05 9/23 Th  06 06 (pdf) HW2 (pdf)
4 06 9/28 T  07 07 (pdf)
7 One-dimensional search methods: Golden section search, Newton's method, secant method. 07 9/30 Th 7  08 08 (pdf) 　
5 8  

Part II: Unconstrained Optimization

 Multi-dimensional algorithms. Gradient methods: form, steepest descent, convergence. Chs. 1-7 Quiz #1 　 　 8  09 09 (pdf) 　
08 10/5 T HW#4 (due 10/11) 　 　
　 Gradient methods: convergence of fixed step size algorithm, steepest descent algorithm. Order of convergence. 09 10/7 Th 　 　  10 10 (pdf) 　
6 9 　 Newton's method: form, order of convergence. Properties of general algorithms. 10 10/12 T HW#5 (due 10/27) 　 　 9  11 11 (pdf) 　
　 　   　 　 1st Midterm Exam 1st Midterm Exam 　 Coverage: Chs. 1-9 1st Midterm Exam
10 　 Conjugate direction methods: form, properties, conjugate gradient formulas. 11 10/14 Th 　 　 10  12 12 (pdf)  
7 11 　 Quasi-Newton methods: form, properties, rank one formula. 12 10/19 T 　 　 11  13 13 (pdf) 　
　 Quasi-Newton methods: DFP and BFGS. 13 10/21 Th  14 14 (pdf) HW3 (pdf) + additional 4 proofs due on 　
8 　 14 10/26 T
　 15 10/28 Th Midterm Exam　
9 12 　 Least squares problems: basic properties, grammian, examples.

RLS. Parameter identification.

 16 11/2 T HW#6 (due 11/10) 　 　 12  15 15 (pdf) 　
　 17 11/4 Th  16 16 (pdf) 　
10

13

 　 Neural network, Backpropagation algorithm, 18 11/9 T HW#7 (due 11/25) 　 　 13  17 17 (pdf) 　
　 19 11/11Th
　 　 　 Randomized search. Simulated annealing.  not covered this semester 　 　 14
11 20

Part IV: Constrained Optimization

 General equality constraints: basic form, example. Lagrange condition for scalar equality constraint. General multivariable Lagrange condition. Tangent and normal space. Minimizing quadratic subject to linear constraint (quadratic programming). Simple linear quadratic regulator problem. Second order conditions. 20 11/16 T 　 　 　 20  29 29 (pdf) HW7 (pdf) due on Final Exam day
21 11/18 Th 　
21 General equality and inequality constraints: form, example. KKT condition with only inequality constraints. Examples. KKT condition for equality and inequality constraints. SONC. SOSC. 22 Chs. 11, 12, 13,  20 Quiz #2 　 　 21  30 30 (pdf)  
11/23 T HW#8 (due 11/29)
23 11/25 Th
12 22 Convex optimization problems. 24 12/2 Th HW#9 (due 12/9) 　 　 22

bv4

  
25 12/7Ta

12/7Tb
13 CVX

Ch. 5

Largrangian and Duality

 Lagrangian, Lagrangian dual function, Weak and strong duality, 26 12/9Th

12/9Thb

 　 　 　 　

bv5

　

  24 24 (pdf) 　
Primal and dual problems, Slater's constraint qualification,  not covered this semester 　 　  25 25 (pdf) 　
Complementary slackness, KKT conditions, KKT conditions for cvx problem, 　 　  26 26 (pdf) 　
Perturbation and sensitivity analysis 　 　  27 27 (pdf) HW6 (pdf) due on
 

14

 　

Part III: Linear Programming

 Constrained optimization: basic form with equality and inequality constraints. Intro to linear programs, geometric view, standard form. linear programming: converting to standard form. 27 12/14 T15 　 　 　 15  19 19 (pdf) 　
　 Linear equations, elementary row operations, basic solutions, basic feasible solutions. Fundamental theorem of LP. 　  20 20 (pdf) 　
　 　 　 2nd Midterm Exam 2nd Midterm Exam 　 Coverage: Chs. 1-15, 23, Part of 16 2nd Midterm Exam
　 Pivoting, changing bases and canonical augmented matrix. Moving from one BFS to an adjacent BFS. How to choose p? How to choose q? Unbounded feasible set 12/14 T16 　 　 　 16  21 21 (pdf) 　
　 　  22 22 (pdf) 　
　 Reduced cost coefficients. Simplex algorithm. Matrix form of simplex. Artificial problem and feasibility. Two phase algorithm. 　  23 23 (pdf) HW5 (pdf) due on
　 Duality in LP: form, example.Weak duality lemma, duality theorem, duality and Simplex algorithm, complementary slackness. Equivalence of feasibility and LP problems. 28a

28b

 12/16 Tha

12/16 Thb

 　 　 　

17

  28 28 (pdf) 　
　

Multi-Objective Optimization

 Vector or Multi-objective optimization, Optimal and Pareto optimal points, Scalarization, Examples: capacity region, sum capacity, etc. 　 　 　 24

 31

31 (pdf) 　

Miscellaneous

Discrete optimization: Integer LP

Variational problems: Brachistochrone, Calculus of variations, Optimal control

 　 　 　 　 　

25

 　
　 　 　 Covers CMs 1-31. Final Exam Final Exam Final Exam

　

 Covers CMs 1-31. Final Exam

　

　

　

　

　

　

Week  Ch.# of 4th Ed. Topic 21-2 20-1 17-2 17-1
 Videos Onenotes  HWs Handouts  Videos pdfs HWs
1 　 Syllabus, Introduction 01 9/7 T 　 　 　 00  01 01 (pdf) 　
　 Examples. 02 9/9 Th 　 　 　 00  02 02 (pdf) 　
1, 2

Part I: Mathematical Review

 Math preliminary: Notation, real vector spaces, linear independence, matrices. HW#1 (Due on Fri. 9/24) 　 　 2-5  03 03 (pdf) HW1 (pdf)
2 3, 4, 5 Math preliminary: Inner product, norm. Eigenvalues and eigenvectors, quadratic forms, calculus of several variables, chain rule, Taylor series, gradient, level sets, directional derivative. 03 9/14 T  04 04 (pdf) 　
6 Definition of optimization problem and types of solutions. Quadratic problems. FONC. 04 9/16 Th HW#2 (Due 10/4) 　 　 6  05 05 (pdf)  
3 SONC and SOSC. Basic iterative algorithms: form, basic properties, line search. 05 9/23 Th  06 06 (pdf) HW2 (pdf)
4 06 9/28 T  07 07 (pdf)
7 One-dimensional search methods: Golden section search, Newton's method, secant method. 07 9/30 Th 7  08 08 (pdf) 　
5 8  

Part II: Unconstrained Optimization

 Multi-dimensional algorithms. Gradient methods: form, steepest descent, convergence. Chs. 1-7 Quiz #1 　 　 8  09 09 (pdf) 　
08 10/5 T HW#4 (due 10/11) 　 　
Gradient methods: convergence of fixed step size algorithm, steepest descent algorithm. Order of convergence. 09 10/7 Th 　 　  10 10 (pdf) 　
6 9 Newton's method: form, order of convergence. Properties of general algorithms. 10 10/12 T HW#5 (due 10/27) 　 　 9  11 11 (pdf) 　
　   　 　 1st Midterm Exam 1st Midterm Exam 　 Coverage: Chs. 1-9 1st Midterm Exam
10 Conjugate direction methods: form, properties, conjugate gradient formulas. 11 10/14 Th 　 　 10  12 12 (pdf)  
7 11 Quasi-Newton methods: form, properties, rank one formula. 12 10/19 T 　 　 11  13 13 (pdf) 　
Quasi-Newton methods: DFP and BFGS. 13 10/21 Th  14 14 (pdf) HW3 (pdf) + additional 4 proofs due on 　
8 14 10/26 T
15 10/28 Th Midterm Exam　
9 12 Least squares problems: basic properties, grammian, examples.

RLS. Parameter identification.

 16 11/2 T HW#6 (due 11/10) 　 　 12  15 15 (pdf) 　
17 11/4 Th  16 16 (pdf) 　
10

13

 Neural network, Backpropagation algorithm, 18 11/9 T 　 　 　 13  17 17 (pdf) 　
19 11/11Th 　
　 　 Randomized search. Simulated annealing. 　 　 　 　 　 14
　 　 Genetic algorithm: Intro, representation schemes, selection. evolution (crossover & mutation). Algorithms for constrained problems: projection, penalty method.  not covered this time 　 　 14,23  18 18 (pdf) HW4 (pdf) due on
　

Part III: Linear Programming

　

 Constrained optimization: basic form with equality and inequality constraints. Intro to linear programs, geometric view, standard form. linear programming: converting to standard form. 　 　 　 　 　 15  19 19 (pdf) 　
　 　 Linear equations, elementary row operations, basic solutions, basic feasible solutions. Fundamental theorem of LP.  not covered this time  20 20 (pdf) 　
　 　 　 2nd Midterm Exam 2nd Midterm Exam 　 Coverage: Chs. 1-15, 23, Part of 16 2nd Midterm Exam
  　 Pivoting, changing bases and canonical augmented matrix. Moving from one BFS to an adjacent BFS. How to choose p? How to choose q? Unbounded feasible set 　 　 16  21 21 (pdf) 　
　 　  22 22 (pdf) 　
　 　 Reduced cost coefficients. Simplex algorithm. Matrix form of simplex. Artificial problem and feasibility. Two phase algorithm.  23 23 (pdf) HW5 (pdf) due on
　 　

Largrangian and Duality

 Lagrangian, Lagrangian dual function, Weak and strong duality, 25 12/2 Th 　 　 　 19  24 24 (pdf) 　
　 　 Primal and dual problems, Slater's constraint qualification, 　 　 　  25 25 (pdf) 　
13 　 Complementary slackness, KKT conditions, KKT conditions for cvx problem,      26 12/7T 　 　 　  26 26 (pdf) 　
　 Perturbation and sensitivity analysis      27 12/9Th 　 　 　  27 27 (pdf) HW6 (pdf) due on
14 　

Part III: Linear Programming

 Duality in LP: form, example.Weak duality lemma, duality theorem, duality and Simplex algorithm, complementary slackness. Equivalence of feasibility and LP problems.      28 12/14 T 　 　 　

17

  28 28 (pdf) 　
11 20

Part IV: Constrained Optimization

 General equality constraints: basic form, example. Lagrange condition for scalar equality constraint. General multivariable Lagrange condition. Tangent and normal space. Minimizing quadratic subject to linear constraint (quadratic programming). Simple linear quadratic regulator problem. Second order conditions.      20 11/16 T 　 　 　 20  29 29 (pdf) HW7 (pdf) due on Final Exam day
     21 11/18 Th 　
21 General equality and inequality constraints: form, example. KKT condition with only inequality constraints. Examples. KKT condition for equality and inequality constraints. SONC. SOSC.      22 Chs. 11, 12, 13,  20 Quiz #2 　 　 21  30 30 (pdf)  
11/23 T 　
     23 11/25 Th 　
12 22 Convex optimization problems.      24 11/30 T   　 　 22  
　 　

Multi-Objective Optimization

 Vector or Multi-objective optimization, Optimal and Pareto optimal points, Scalarization, Examples: capacity region, sum capacity, etc. 29 12/16 Th 　 　 　 24

 31

31 (pdf) 　
　

Miscellaneous

Discrete optimization: Integer LP

Variational problems: Brachistochrone, Calculus of variations, Optimal control

 　 　 　

25

 　
　 　 　 Covers CMs 1-31. Final Exam Final Exam Final Exam

　

 Covers CMs 1-31. Final Exam