Course Syllabus

Instructor:

Professor Chandrajit Bajaj

  • Lecture Hours – Mon, Wed- 3:30 - 5:00 pm. GDC 2.210 or Zoom
  • Office hours -- Tue 1:00 p.m. - 3:00 p.m. or by appointment ( Zoom or POB 2.324)
  • Contact: bajaj@oden.utexas.edu

NOTE: All questions related to class should be posted through Piazza. Here is the link to register for Piazza: You can also join via the Piazza Tab on the Canvas course page.

Teaching Assistant

Omatharv Vaidya

  • Office hours – Tue/Thurs 10:00 a.m. - 11:00 a.m. GDC 1.302 - TA station Desk 2 or Zoom
  • Contact: vomatharv@gmail.com

Note: Please attempt to make reservations a day before to avoid conflicts. 

Course Motivation and Synopsis

This fall course is on foundational mathematical, statistical, and computational learning theory and the application of data-learned predictive models. Students shall be exposed to modern machine learning approaches in optimized decision-making and multi-player games involving stochastic dynamical systems and optimal control. These latter topics are foundational to the training of multiple neural networks (agents) both cooperatively and in adversarial scenarios helping optimize the learning of all the agents.

An initial listing of lecture topics is given in the syllabus below. This is subject to modification, given the background and speed at which we cover ground.  Homework exercises shall be given almost bi-weekly.  Assignment solutions that are turned in late shall suffer a 10% per day reduction in credit and a 100% reduction once solutions are posted. There will be a mid-term exam in class. The content will be similar to the homework exercises. A list of topics will also be assigned as take-home final projects to train the best of machine-learned decision-making (agents). The projects will involve ML programming, an oral presentation, and a written report submitted at the end of the semester.

This project shall be graded and be in lieu of a final exam.

The course is aimed at graduate students. Those in the 5-year master's program students, especially in the CS, CSEM, ECE, STAT, and MATH., are welcome. You’ll need algorithms, data structures, numerical methods, and programming experience (e.g., Python ) as a CS senior, mathematics and statistics at the level of CS, Math, Stat, ECE, plus linear algebra, computational geometry, plus introductory functional analysis and combinatorial and numerical optimization (CS, ECE, CSEM, Stat, and Math. students).  

Course Reference Material (+ reference papers cited in lectures )

  1. [B1] Chandrajit Bajaj (frequently updated)  A Mathematical Primer for Computational Data Sciences 
  2. [M] Kevin Murphy Machine Learning: A Probabilistic Perspective
  3. [MU] Michael Mitzenmacher, Eli Upfal Probability and Computing (Randomized Algorithms and Probabilistic Analysis)
  4. [SB] Richard Sutton, Andrew Barto Reinforcement Learning
  5. [SD] Shai Shalev-Shwartz, Shai Ben-David Understanding Machine Learning, From Theory to Algorithms
  6. [Basar] Tamer Basar  Lecture Notes on Non-Cooperative Game Theory.
  7. [BHK] Avrim Blum, John Hopcroft, and Ravindran Kannan. Foundations of Data Science
  8. [PML1] Kevin Murphy Probabilistic Machine Learning: An Introduction.
  9. [PML2] Kevin Murphy Probabilistic Machine Learning: Advanced Topics.
  10. [BV] Stephen Boyd and Lieven Vandenberghe Convex Optimization.
  11. Extra reference materials.

TENTATIVE  COURSE OUTLINE (in Flux). 

Date Topic Reading Assignments

Mon

08-21-2023

1. Introduction to High-Dimensional Spaces, Belief, and Decision-Making Spaces, [Lec1]

Dynamical Systems and Deep Learning [notes]

Modern Statistical Machine Learning [notes]

[M] Ch 1.1, 1.2, 1.3

[SB] Ch1

 

 

Wed

08-23-2023

2. Learning Stochastic Regression Models  [Lec2]

Geometry of Norms and Approximations [notes];

Log-Sum-Exponential-Stability [notes]

[SD]

[M] Ch 1

[A1] with [latex template] out today; due by 09-06-2023, 11:59 pm

Mon

08-28-2023

3. Learning Theory and Model Selection [Lec3]

PAC learning, Complexities [notes]

Probability, Information and Probabilistic Inequalities [notes]

 

[M] Ch 1.4.7, 1.4.8

[MU] Ch 1-3

[B1] Appendix

[PML1] 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.2, 4.7. 

 

Wed

08-30-2023

4. Stocashtic Machine Learning: Entropy, Distributional Estimates [Lec 4] 

 

[MU] Chap 4, 24.2

[BHK] Chap 12.4,12.6

[PML1] Chap 4.1, 4.2, 4.5, 4.7, 6.1, 6.2.

[PML2] Chap 3.3, 5.1, 5.2, 15.1, 15.2, 15.3

 

Wed

09-06-2023

5. Sampling in High-Dimensional  Space-Time 2:    [Lec-part1]  Concentration of Measure  [notes]

[SD] Chap 12

[A1] due by midnight.

[A2] will be out on 09-10-2023; due by 09-24-2023, 11:59 pm.

 

Mon

09-11-2023

6. Sampling in High Dimensional Space-Time 3 : MonteCarlo vs Quasi Monte-Carlo, Relationship to Integration Error H-K Inequality [Lec4-part1][Lec4-part2]

 

[PML1] 3.6

[PML2] 2.6, 4, 7.4.5, 7.4.6, 12.1, 12.2, 12.3, 12.6.

 

Wed

09-13-2023

7. Statistical Machine Learning 1: Introduction to Markov Chains, Page Rank, MCMC [Lec-notes, notes2]

[BHK] Chap 4

[MU] Ch 7, 10

 

 

Mon

09-18-2023

8. Statistical Machine Learning 2: Sampling and Learning  with MCMC Variations [Lec7]

 

 

[SD] Chap 24

[BV] Chap 1-5

Wed

09-20-2023

9. Statistical Machine Learning 3:  Bayesian Inference with MCMC and Variational Inference [notes]

Learning by Random Walks on Graphs  [notes-BHK]

[BHK] Chap 2.7

[SD] Chap 23, 24

[A2] due by 09-24-2023, 11:59 pm.

Mon

09-25-2023

10.  Learning SVM via Continuous Stochastic Gradient Descent Optimization [notes]

 Learning with  SGD variations, Adagrad, RMSProp, Adam, ...] [notes]

Continuous Stochastic Gradient Descent (SGD)  -- Simulated Annealing, Fokker-Planck [notes]

 [M]  Ch 9, 13

Wed

09-27-2023

11.  Non-convex Optimization: Projected Stochastic Policy Gradient [Notes]

 

[M]  Ch 2, 5

[A3] will be out on 09-28-2023; due by 10-12-2023, 11:59 pm.

Mon

10-02-2023

12. Random Projections, Johnson-Lindenstrauss, Compressive Sensing, [notes] 

Tensor Sketching  in Space-Time [notes2]

[M]  Ch 4

 

Wed

10-04-2023

13. Robust Sparse Recovery; Alternating Minimization  [notes2]

 

[M]  Ch 15

[M] Chap 11

[PML1] 3.2, 3.3, 17.2

[PML2] 2.3, 18, 23, 35.4.3

 

 

Mon

10-09-2023

14: Statistical Machine Learning 4: Learning Models with Latent Variables / Expectation Maximization [notes]

 [M]  Ch 15

 

 

Wed

10-11-2023

 15.  Statistical Machine Learning 5: Multivariate Gaussians and Gaussian Processes [notes1] [notes]  [notes2]

 

 [BHK] Chap 5

[A3] due by 10-12-2023, 11:59 pm.

Mon

10-16-2023

16.  Statistical Machine Learning V: Non-Gaussian Processes, Conjugate Priors  [notes]

 

[M] Chap 14

Wed

10-18-2023

 MIDTERM examination

 

 

 

 

Mon

10-23-2023

17. Learning Dynamics,  Lyapunov Stability  and connections to Training Deep Networks [notes]

Learning Dynamics:  Auto_Regressive Machine Learning  [notes]

 

 

 

A4} will be out today; due by 11-07-2023, 11:59 pm.

 

Wed

10-25-2023

 18. Learning Dynamics with Neural ODEs (NODEs): Adjoint Method for BackProp [notes]

 Implicit Euler, Convergence [notes]

Non-convex Projected Gradient Descent [notes-references]

 

Mon

10-30-2023

19.  Introduction to Stochastic Processes [pdf]

Learning Dynamics with Stochastic Processes [notes] 

Learning Dynamics with Stochastic Neural ODEs (SNODEs) : Stochastic Adjoint Methods I [notes] [notes2]

See references cited in notes

 

 

 

Wed

11-01-2023

20.Learning Dynamics with Control and Optimality [notes]

See references cited in notes

 

Mon

11-06-2023

 

21. The role of Sensors and Optimal Sensor Fusion:

Basics of Kalman Filters [notes]

Illustrated Kalman Filters [notes]

 

 

Project details will be out on 11-07-2023; Part (I) of the project due by 11-30-2023, 11:59 pm.

Wed

11-08-2023

22.  Reinforcement Learning 2: Learning Dynamics with Optimal Control:  Dynamics  LQR, iLQR, iLQG [notes] 

 

See references cited in notes and paper

 

[A4] due today, i.e., 11-08-2023, 11:59 pm.

 

 

Mon

11-13-2023

 23. Reinforcement Learning 3: Guided Policy Search [notes]

 See references cited in [notes]

Wed

11-15-2023

24.   RL 3:   Bandit Algorithms, Thompson Sampling [notes]

Statistical Machine Learning 6: Bandit Algorithms [notes] 

 

 

See references cited in [notes]

Part 1 of the project due by 11-30-2023, 11:59 pm. 

Project video presentation due by 12-12-2023; 11:59 pm.

Mon

11-27-2023

25.  Time Series Analysis [notes] [notes]

 

Game-Theoretic Learning 1:  MARL -Markov Games  [notes].

Games & MARL  II [notes]

 

 

 

 

 

 

Wed

11-29-2023

26. Reward Reshaping: Inverse Reinforcement Learning with Optimal Control [notes]

Game-Theoretic Learning 1:  MARL -Markov Games  [notes].

Games & MARL  II [notes]

Game Theoretic Learning 2: Stackelberg Equilibrium [notes]

Fri

12-04-2023 onwards

27.  Active Learning 2:  Dynamic POMDPS

Diffusion Models with Stochastic Langevin Dynamics [notes]

Energy-Based Learning: Hopfield Networks, Boltzmann Machines, Restricted Boltzmann Machines. [notes]

 

Final Project video presentation due by 12-12-2023; 11:59 pm.

Addtl. Material

 NeuralPMP: Reinforcement Learning with Stochastic Hamiltonian Dynamics, Pontryagin Maximum Principle [arxiv]

 [Basar] See Lectures 1, 2, 3 

Final report due by 12-12-2023; 11:59 pm.

Addtl. Material

 

Normalzing Flows [notes]

Actionable Learning [notes]

Robust Continuous learning of PDEs using Sparse Gaussian Processes [arxiv]

 

Markov Decision Process (MDPs)  and  Markov Games -- [notes]

Energy-Based Learning: Hopfield Networks, Boltzmann Machines, Restricted Boltzmann Machines. [notes]

Actionable Learning [notes]

Spectral Methods for Learning Dimension Reduction -KPCA, Eigen- Fischer-Faces[notes] [notes]   E.  KSVM [Notes], Fischer LDA, KDA [notes] 

Statistical Machine Learning : (a) Separating Mixture of Gaussians   [notes]  (b) Expectation Maximization   [notes]

 Some important Classical Machine Learning Background.

Addtl. Material

Robustness Guarantees for Bayesian Inference and Gaussian Processes [paper]

Risk Averse No Regret Learning for Convex Games [paper]

RL 4: Markov (Reward, Decision) Processes: MPs, MRPs, MDPs and POMDPs [notes]

 Statistical Machine Learning 3: Bayesian Inference and Generative Models (VAEs and GANs) [notes1]

Bayesian Modelling and Inference [notes2]

Connections to Variational AutoEncoders (VAEs) [notes]

Statistical Machine Learning 4: Transform Sampling revisited, Sampling Non-Linear Probability Distributions [notes].   Generative Adversarial Networks [notes]  

Learning with Normalizing Flows [notes]

Some Theoretical Bounds on Bayesian Optimization and Reinforcement Learning.

 

 

Project FAQ

1. How long should the project report be?

Answer: See directions in the Project section in assignments. For full points, please address each of the evaluation questions as succinctly as possible. You will get feedback on your presentations, which should also be incorporated into your final report.

Assignments, Exam, Final Project, and Presentation

There will be four take-home bi-weekly assignments, one in-class midterm exam, one take-home final project (in lieu of a final exam), and one presentation based on your project progress. The important deadline dates are:

  • Midterm: Wednesday, October 18, 3:30 pm - 5:00 pm.
  • Final Project Written Report Part 1: Due November 30th, 11:59 pm.
  • Final Project Written Report, and Presentation Video, Due December 12th, 11:59 pm

Assignments

There will be four written take-home HW assignments and one take-home final project report. Please refer to the above schedule for assignments and the final project report due time.

Assignment solutions that are turned in late shall suffer a 10% per day reduction in credit and a 100% reduction once solutions are posted.

Course Requirements and Grading

Grades will be based on these factors:

  • In-class attendance and participation (5%)
  • HW assignments (50% and with the potential to get extra credit) 

4 assignments. Some assignments may have extra questions for extra points you can earn. (They will be specified in the assignment sheet each time.)

  • In-class midterm exam (15%) 
  • First Presentation & Report (10%)
  • Final Presentation & Report (20%)  

Students with Disabilities. Students with disabilities may request appropriate academic accommodations from the Division of Diversity and Community Engagement, Services for Students with Disabilities, 471-6259, http://www.utexas.edu/diversity/ddce/ssd . 

 

Accommodations for Religious Holidays. By UT Austin policy, you must notify the instructor of your pending absence at least fourteen days prior to the date of observance of a religious holiday. If you must miss a class or an examination in order to observe a religious holiday, you will be given an opportunity to complete the missed work within a reasonable time before or after the absence, provided proper notification is given.

 

Statement on Scholastic Dishonesty. Anyone who violates the rules for the HW assignments or who cheats on in-class tests or the final exam is in danger of receiving an F for the course. Additional penalties may be levied by the Computer Science department, CSEM, and the University. See http://www.cs.utexas.edu/academics/conduct/