Fa20 - FDTNS OF PREDICTIVE MACHN LRNG (51426)

Fa20 - FDTNS OF PREDICTIVE MACHN LRNG (51426)

Instructor:

Professor Chandrajit Bajaj

NOTE: Most questions should be submitted to Canvas rather than by sending emails to the instructor. Please attempt to make reservation a day before for the office hour  to avoid conflicts. 

 

Teaching Assistant

Huancheng Chen

Note: Please attempt to make reservations a day before for the office hours  to avoid conflicts. 

Course Motivation and Synopsis

This course shall dwell on foundational aspects of predictive machine  learning  and particularly from   time series data through modeling of dynamical systems and optimal control.    You will learn foundational intertwined topics of mathematics, computer science and statistics, namely, Bayesian estimation,  variational inference, stochastic sampling, compressive measurements, convex and non-convex geometric, manifold optimization,  stochastic optimal control, dimensionality reduction, stability, predictions with uncertainty quantification, dynamic programming, Gaussian processes,  Markov decision processes,  Kalman filtering, optimal transport, normalizing flows, invertible diffeomorphisms, and more !  You will also learn deep  reinforcement learning techniques for  spatio-spectral-temporal predictions,  recommendations, forecasting , counterfactual  estimations and robust decision making. Deep networks that are often utilized for these  include recurrent,  residual and reservoir nets with variations of OdeNets and PdeNets, to create variational auto-encoders, and  generative adversarial predictors.  Application data stem from a variety of  sensing and simulated data (sequence,  time-series,  multi-hyper-spectral video, molecular dynamics, materials  and fluid dynamics), satisfying physical and natural conservation laws.  Issues of measurement and computation errors, noise and outliers shall be central to bounding the precision  and accuracy of the data analysis. The course is aimed at graduate students. Those in the 5-year master's program students, especially in the CS, CSEM, ECE, STAT and MATH. are welcome. You’ll need mathematics, computer science and statistics at the level of first year graduate,  linear algebra, computational geometry, probability and statistics plus introductory functional analysis and numeric optimization  (e.g., for  CS and ECE students) and combinatorial optimization (e.g.,for  CSEM and Math. students).  

Course Material.

  1. [B1] Chandrajit Bajaj (frequently updated)  A Mathematical Primer for Computational Data Sciences
  2. [MRT] Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar Foundations of Machine Learning
  3. [SB] Richard Sutton, Andrew Barto , Reinforcement Learning ; An introduction
  4. [KM] Kevin Murphy  A Probabilistic Perspective
  5. [MU] Michael Mitzenmacher, Eli Upfal Probability and Computing (Randomized Algorithms and Probabilistic Analysis)
  6. [GBC] Ian Goodfellow, Yoshua Bengio, Aaron Courville Deep Learning
  7.  [KW] Diederik P. Kingsma and Max Welling An Introduction to Variational Auto Encoders
  8. [RW] Carl Rasmussen and Chris Williams. Gaussian Processes for Machine Learning
  9. [BHK] Avrim Blum, John Hopcroft and Ravindran Kannan. Foundations of Data Science
  10. [JK] Prateek Jain, Purshottam Kar Non-Convex Optimization for Machine Learning .
  11. [BV-CVX] Stephen Boyd, Lieven Vandenberghe. Convex Optimization
  12. [MJ] Mathew James  Nonlinear Control Systems

 

TENTATIVE  COURSE OUTLINE (in Flux). 

Date Topic Reading Assignments

Wed

08-26-2020

1. Predictive Machine Learning  I 

Supervised, Unsupervised, Reinforcement

Time-Series Applications [notes]

[KM] Chap 1

[MRT] Chap 1-6

[SB] Chap 5

 

Mon

08-31-2020

2. Predictive Machine Learning  II 

Model Selection, Capacity, Overfitting, Underfitting [notes]

Parametric vs Non-Parametric (see readings)

Multi-Layer  Perceptron Learning (see readings)

Projected Gradient Descent [notes]

[MRT] Chap 4 

[KM] Chap  1-2

[GBC] Chap 5

[A1] posted

Wed

09-2-2020

3. Predictive Machine Learning  III  (see readings)

SVD, Kernel SVM [notes]

Optimization I: Lagrange Multipliers, KKT  [notes]

Probabilistic  Techniques I: Distributions, Inequalities [notes]

[KM] Chap  3

[GBC] Chap 5

Wed

09-09-2020

4.  Optimization II: Convex and Non-Convex Optimization, Projected  gradient descent,  Matrix Completion,  [notes]

Sub-Gradients and Proximal Gradients [notes]

Probabilistic Techniques II [notes]

Bayesian Concept Learning, MLE, MAP

[BV-CVX] Chap 1 -5

[KW] A.2.1

[GBC] Chap 5

[JK] Chap 2

 

Mon

09-14-2020

5. Probabilistic Techniques III 

Sampling, MonteCarlo Sampling [notes]  Low Discrepancy Quasi-Monte Carlo Sampling, [slides], Integration Error H-K Inequality [notes]

[KM] Chap  3

See Refs in Notes

[A1] due day 

Wed

09-16-2020

6.  Probabilistic Techniques IV

Transformations Sampling [notes1] [notes2]

Normalizing Flow I [notes]

[SB] Chap  5

See Refs in Notes in Slides

[A1] solution 

[A2] out today

Mon

09-21-2020

7.  Sampling Multivariate Gaussians in High Dimensions, Separating Mixture of Gaussians I [notes]

[BHK] Chap 3

See Refs in notes

 

Wed

09-23-2020

8. Statistical Machine Learning I : Separating Mixture of Gaussians II, Expectation Maximization I  [notes]

[KM] Chap 11

[B1] Chap 10

Mon

09-28-2020

9. Expectation Maximization II (Latent Variable Models, Soft Clustering, Mixed Regression) [notes]

Connections to Generative Models [notes]

[KM] Chap 9

[KM] Chap 11-12

 

Wed

09-30-2020

10. Variational Inference:  State Space Representations  [Slides]

Inference  and Generative Model Training [slides]

KL, JS, Wasserstein, Bregman Divergences [notes]

[KW] Chap 3, 4

See Refs in Notes

[A2] due day 

[A3] out 

on Oct. 2

Mon

10-05-2020

11. Variational Auto Encoders I [KW]

Optimization II: Stochastic Gradient Descent, Fokker Planck equation [notes]

 

[KW] Chap 2

See Refs in Notes

[A2] solution

 

Wed

10-07-2020

12. Variational Auto Encoders II: Mixture of Gaussian Priors [Slides]

Markov Chain Monte Carlo Sampling of Unknown  Posterior Distributions [notes]

 

[KW] Chap 2, 3

 

[BHK Chap 4]

See Refs in Notes

 

 

Mon

10-12-2020

 13. Dynamical Systems and Non-linear Predictive Control [see JM]

Linearizing  Dynamics I [notes]

 

[JM] Nonlinear Control Chap

[KM] Chap 21-22

[Boyd] Slides

 

Wed

10-14-2020

14: Dynamical Systems and Non-linear Predictive Control II [chap]

Connections to Residual Nets, Ode Nets [notes1, notes 2]

[JM] Nonlinear Control Chap

 

[A3]  due on Oct.16

Mon

10-19-2020

15. Dynamical Systems and Non-linear Predictive Control III

Koopman Theory, Dynamic Mode Decomposition, with Applications to Reduced Models [notes] [slides]

 

[A3]  solution on Sunday Oct 18 

Wed

10-21-2020

 

MIDTERM in Class [Online]

[A4] out today

Mon

10-26-2020

16. Stochastic Gaussian Processes I : Uncertainty Quantification [paper], [RWbook]

 

[RW]  Chap 1-3

Reference paper

 

Wed

10-28-2020

 17.  Stochastic Gaussian Processes II: Uncertainty Propagation, Kalman Filtering [notes][KF-Dynam-paper]

Reference KF-Dynam paper

 

Mon

11-02-2020

18. Matrix and Tensor Sketching [notes]; SketchCoreSVD

Robust Sparse Recovery; Alternating Minimization[note], Connections to Compressive Sensing , L0, L1 minimization [notes2]

 

Reference paper

 

[JK] Chap 7-8

project topics out 

Wed

11-04-2020

19.Geometry of Deep Reinforcement Learning  I : 

Adaptive Time Series Forecasting

 

[GBC] Chap 10

See Refs in chap

 

[A4]  due day

solution out on Nov. 6

[A5] out on Nov. 6

 

Mon

11-09-2020

 20. Geometry of Deep Reinforcement  Learning II: Continuous Dynamical Systems

 

[GBC] Chap 10

See Refs in Notes

project topic and group decision due day

Wed

11-11-2020

21. Geometry of Deep  Reinforcement Learning III:  Generative Models

[BHK] Chap 5

[KW] Chap 2

[GBC] Chap 20

See Refs in Notes

Mon

11-16-2020

22. Geometry of Deep Reinforcement Learning  IV  

 Optimal Control for Unknown Dynamical Systems

[GBC] Chap 10-12

[SB] Chap 4

Part 1 of Project Due

 

Wed

11-18-2020

23. Geometry of Deep Reinforcement  Learning V:   

Robust Optimal Control for Unknown Dynamical Systems

[GBC] Chap 10-12

See Refs in Notes

[A5] due on Nov. 20

Solution out on Nov. 22

Mon

11-23-2020

 24.  Geometry of Deep  Reinforcement Learning VI: Value Iteration 

[GBC] Chap 10-12

[SB] Chap 6

Mon

11-30-2020

25. Geometry of Deep Reinforcement Learning IV: Policy Gradients

See Refs in Notes


Wed

12-2-2020

 

26. Geometry of Deep Reinforcement Learning IV:  Actor-Critic Methods

See Refs in Notes

 

Mon

12-07-2020

27.  Geometry of Deep Learning IV: Inverse RL& Shape Optimization

 See Refs in Notes

 

 

THUR

12-10-2020

Presentations (5p - 7pm)

 

12-14-2020

FINAL REPORTS DUE

 

Final Project Report Due 

 

Project FAQ

1. How long should the project report be?

Answer: See directions in the Class Project List.  For full points, please address each of the evaluation questions as succinctly as possible. Note the deadline for the report is Dec 11 midnight. You will get feedback on your presentations,  that should also be incorporated in your final report.

Tests

There will be one in-class midterm exam and one final project. The important deadline dates are:

  • Midterm: Monday Oct 21, 2020
  • Final Project  Written Report, Part 1 Due: Nov 16, 2020 and Part 2 (completed) Due: Dec 14, 2020

 

Assignments

There will be five written homework (HW) assignments and one final project. Each HW roughly spans 2 weeks. There will be a 2-day grace beyond the due date of the HW, before solutions are released. You will be penalized 5%, each day you are late. After You are allowed to turn in your HW after solutions are released. However  you will be penalized 50%,  and then 5% additional day of delay. Please remember HW should be in your own words,  and not copied, to demonstrate learning and understanding .  The Final project report should be turned in two parts. The first part should have a clear statement of  the problem you have selected to work on,  the name of your partner (if doing this jointly) and  the prior papers whose approach  your basing your solution. Part 2 (and final) project report submission should build upon Part 1,  include details of your solution, results obtained and a discussion of achievements/future work.   Please also refer to the above schedule for all assignments and final project report due time.

Course Requirements and Grading

Grades will be based on these factors

  • In-class attendance and participation (5%)
  • HW assignments (50% and with potential to get extra credit) 

5 assignments. Some assignments may have extra questions for extra points you can earn. (They will be specified in the assignment sheet each time.)

  • In-class midterm exam (15%) 
  • First Presentation & Report (10%)
  • Final Presentation & Report (20%)  

Students with Disabilities. Students with disabilities may request appropriate academic accommodations from the Division of Diversity and Community Engagement, Services for Students with Disabilities, 471-6259, http://www.utexas.edu/diversity/ddce/ssd . 

 

Accommodations for Religious Holidays. By UT Austin policy, you must notify the instructor of your pending absence at least fourteen days prior to the date of observance of a religious holiday. If you must miss a class or an examination in order to observe a religious holiday, you will be given an opportunity to complete the missed work within a reasonable time before or after the absence, provided proper notification is given.

 

Statement on Scholastic Dishonesty. Anyone who violates the rules for the HW assignments or who cheats in in-class tests or the final exam is in danger of receiving an F for the course. Additional penalties may be levied by the Computer Science department,  CSEM  and the University. See http://www.cs.utexas.edu/academics/conduct/

Course Summary:

Date Details Due