Professor Chandrajit Bajaj

• Lecture Hours – Mon, Wed- 1:00 - 2:30 pm. ETC 2.132. If Online, go to Zoom panel.
• Office hours -- Tue 1:00 p.m. - 3:00 p.m. or by appointment ( Zoom or POB 2.324)
• Contact: bajaj@cs.utexas.edu bajaj@oden.utexas.edu 

NOTE: All questions related to class should be posted through Piazza. Here is the link to register for Piazza: You can also join via the Piazza Tab on the Canvas course page.

Teaching Assistant

Shubham Bhardwaj

• Office hours – Thur 1:00 p.m. - 3:00 p.m. ( Zoom or POB 2.102)
• Contact: shubham.bhardwaj@utexas.edu 

Note: Please attempt to make reservations a day before to avoid conflicts. 

Course Motivation and Synopsis

The Fall Predictive Machine Learning course  will teach you the latest on reinforcement learned , risk averse stochastic decision making process useful in diverse dynamical environments. These stochastic machine learned  trained, verified  and validated on  signals and information, filtered from noisy observation data distributions collected from various multi-scale dynamical systems. The principal performance metrics will be on online and energy efficient training, verification and validation protocols that achieve principled and stable learning for maximal generalizability .  The emphasis will be on  possibly corrupted data and/or  the lack of full information for the learned stochastic decision making dynamic algorithmic  process. Special emphasis will also  be given to the underlying  mathematical and statistical physics principles  of  Free Energy and stochastic Hamiltonian  dyamics . Students shall  thus be exposed to the latest stochastic  machine learning  modeling approaches for  optimized decision-making, multi-player games involving stochastic dynamical systems and optimal stochastic control. These latter topics are foundational to the training of multiple neural networks (agents) both cooperatively and in adversarial scenarios to optimize the learning process of all the agents.

An initial listing of lecture topics and reference material are given in the syllabus below. This is subject to some modification, given the background and speed at which we cover ground.  Homework exercises shall be given almost bi-weekly.  Assignment solutions that are turned in late shall suffer a 10% per day reduction in credit and a 100% reduction once solutions are posted. There will be a mid-term exam in class. The exam content will be similar to the homework exercises. A list of topics will also be assigned as take-home final projects to train the best of scientific machine-learned decision-making (agents). The projects will involve modern ML programming, an oral presentation, and a written report submitted at the end of the semester.

This project shall be graded and be in lieu of a final exam.

The course is open to graduate students in all disciplines. Those in the 5-year master's program students, and in the CS, CSEM, ECE, MATH, STAT, PHYS, CHEM, and BIO, are welcome. You’ll need an undergraduate level  background  in  the intertwined topics of algorithms, data structures, numerical methods, numerical optimization,  functional analysis, algebra, geometry, topology, statistics, stochastic processes . You will need programming experience (e.g., Python ), at  a CS undergraduate senior level.

 

Course Reference Material (+ reference papers cited in lectures )

1. [B1] Chandrajit Bajaj (frequently updated)  A Mathematical Primer for Computational Data Sciences 
2. [PML1] Kevin Murphy Probabilistic Machine Learning: An Introduction.
3. [PML2] Kevin Murphy Probabilistic Machine Learning: Advanced Topics.
4. [M1] Peter S. Maybeck Stochastic Models, Estimation and Control Volume 1
5. [M2] Peter S. Maybeck Stochastic Models, Estimation and Control Volume 2
6. [M3] Peter S. Maybeck Stochastic Models, Estimation and Control Volume 3
7. [MU] Michael Mitzenmacher, Eli Upfal Probability and Computing (Randomized Algorithms and Probabilistic Analysis)
8. [SB] Richard Sutton, Andrew Barto Reinforcement Learning
9. [SD] Shai Shalev-Shwartz, Shai Ben-David Understanding Machine Learning, From Theory to Algorithms
10. [Basar] Tamer Basar  Lecture Notes on Non-Cooperative Game Theory.
11. [BHK] Avrim Blum, John Hopcroft, and Ravindran Kannan. Foundations of Data Science
12. [BV] Stephen Boyd and Lieven Vandenberghe Convex Optimization.
13. [DSML] Qianxiao Li - Dynamical System and Machine Learning
14. Extra reference materials.

TENTATIVE  COURSE OUTLINE (in Flux). 

Date	Topic	Reading	Assignments
	Module 1: Foundations of Stochastic Processes & Dynamical Systems
Mon 08-25-2025	1. Introduction to High-Dimensional Spaces, Belief, and Decision-Making Spaces [Lec1] [colab]	[M1] 1, 2, 3, 4 [DSML] Dynamical Systems and Deep Learning [slides]
Wed 08-27-2025	2.1 From Bayesian Thinking to the Kalman Filter Grounding State Estimation in Bayesian Filtering [Lec2]	[M1] - Ch 3 2.2 Geometry of Norms and Approximations - [notes]	[A1] with [latex template] out today; [style.sty]
Wed 09-03-2025	3. Why Nonlinearity Breaks Our Gaussianity Framework [lect 3]	[M1] Ch 3 [PML2] Ch 18 Probability Primer
Mon 09-08-2025	4.1   Mathematical Foundations of Bayesian Filtering  [lec]	[PML2] Ch 18   4.2 Bayesian Deep Learning -  [notes] 4.3 Univariate Time Series Analysis  - [supp notes]
	Module 2: Sequential Models & Filtering
Wed 09-10-2025	5.1  Sequential Monte Carlo Methods: Making the Intractable Tractable [lec]   Dynamic Mode Decomposition [notes]	5.1 - [Supp notes] 5.2 -  [Supp Notes2][Lec5] 5.3 - [notes]   [PROBABILITY PRIMER 2 -  ADVANCED]
Mon 09-15-2025	6. SDE integration - SGD, SGLD (Stochastic Langevin Gradient Descent) [lec][colab notebook]		[A2] released [A2 pdf] [A2 latex] [tensor.npy]
Wed 09-17-2025	7. HSGLD - Hamiltonian Stochastic Langevin Gradient Descent [lec] [colab notebook]
Mon 09-22-2025	8. : The Friction Knob - Understanding Your Optimizer [lec] [colab notebook]	[BHK] Ch 2.1-2.6 [PML2] Ch 11 6.1 - [notes]
Wed 09-24-2025	9.  The Transport View of Optimization - Why Your Optimizer is Moving Probability Mass [lec]	[PML1] Ch 3.6 [PML2] Ch 2.6, 4.2, 7.4.5, 7.4.6,
	Module 3: Stochastic Optimization & Variational Methods		[A2] due Sep 28 midnight
Mon 09-29-2025	10.  MCMC Foundations - From Random Walk to Hamiltonian Flow. The Journey from Discrete Jumps to Continuous Dynamics [lec]	[PML2] Ch 12.1, 12.2, 12.3 12.6 10.1 -   [notes]
Wed 10-01-2025	11. Statistical Machine Learning 3:  Bayesian Inference with MCMC and Variational Inference	[BHK] Chap 2.7 [PML2] Ch 12.1, 12.2, 12.3 12.6
Mon 10-06-2025	12.   Stochastic Optimization - Connections of  MCMC and VAE	[PML1] Ch 11.4 [PML2] Ch 15.2.6, 28.6.5 12. - [notes]
	Stochastic Optimization Formulations and Statistical Machine Learning   Learning SVM via Continuous Stochastic Gradient Descent Optimization   Continuous Stochastic (Noisy) Gradient Descent (SGD)  -- Simulated Annealing, Fokker-Planck	[PML1] Ch 8.1, 8.2, 8.3, 8.4, 8.5 [PML2] Ch 6.3 10 - [notes] 10.2 - [notes]
	Optimization and Machine Learning 1: KKT, LP, QP, SDP, SGD,	[PML1] Ch 8.6 11 -  [notes]	[A3] will be out on Friday;
	Non-convex Optimization: Projected Stochastic Gradient   Learning with  SGD variations, Adagrad, RMSProp, Adam, ...]	[BHK] Ch 2.7 [PML1] Ch 7 12.1 - [supp-notes]
Wed 10-08-2025
Mon 10-13-2025	14: Random Projections, Johnson-Lindenstrauss, Compressive Sensing,
	Module 4: Manifolds, Hamiltonians, & Learning Dynamics
Wed 10-15-2025	15.  Matrix Sampling and Sketching	[PML1] Ch 17.2 [PML2] Ch 18.1,18.2,18,3, 18.5 [notes]
Mon 10-20-2025	16.   Tensor Sketching  in Space-Time	[PML1] Ch 4.6 [PML2] Ch 3.4, 18.4, 18.6 [M1] Ch 1, 2, 5.8 [notes]
Wed 10-22-2025	12.  Hamiltonians, Symplectic Manifold and Controllable Flows I
Mon 10-27-2025	17.  Stochastic Hamiltonians, Symplectic Manifold and Controllable  Free Energy  Flows II
Wed 10-29-2025	MIDTERM
Mon 11-03-2025	19.  Data Clustering with Hamiltonians and Hamiltonian Dynamics	[TODO] - inaccurate refs [M1] Ch 4, 5.7 [M2] Ch 11, 12
Wed 11-05-2025	20.  Learning Dynamics with Control and Optimality	[M1] Ch 2.5 [M3] Ch 13.1, 13.2, 13.3	[A4] will be out;
Mon 11-10-2025	21. Stochastic Gradient Hamiltonian MC with Controllable Hamiltonian Dynamics	[TODO - incorrect refs] [PML2] Ch 8.1-8.5 [M1] Ch 1.3, 5.1-5.8 [Additional notes]	Project details will be out; [check here for final project]
	Module 5: Reinforcement Learning & Inverse Problems
Wed 11-12-2025	22.  Reinforcement Learning 1: Learning Dynamics with Optimal Control:  Dynamics  LQR, iLQR, iLQG	[M3] Ch 13.4, 13.5, 13.6, 14.1-14.5, 14.13
Mon 11-17-2025	23. Reinforcement Learning 2: Guided Policy Search [Lec23]	[PML2] Ch 35.3, 35.4
Wed 11-19-2025	24.   Reinforcement Learning 3:   Hamiltonian Dynamics, Pontryagin Maximum Principle	[PML2] Ch 34.4 [notes] NeuralPMP  [arxiv]  Safe Pontryagin[arxiv]
Mon 11-24-2025	25.  Reinforcement Learning 4:  25.1 - Policy Gradient Optimization Methods ,  25.2 -Stochastic Hamiltonian Gradient Flows	[PML2] Ch 29.3, 29.7, 29.12 [Basar] See Lectures 1, 2, 3  25.1 - [notes]
Mon 12-01-2025 [Online]	26. Reinforcement Learning 5: Stochastic Hamiltonian Flows I	[PML2] Ch 35.6
Wed 12-03-2025 [Online]	27.  Reinforcement Learning 5: Forward and Inverse Problems, Scientific Discovery	[PML2] Ch 24, 25, 34.5
Addtl. Material	Probability, Information and Probabilistic Inequalities [notes] Log-Sum-Exponential-Stability [notes]	PML1] Ch 4.1, 4.2, 4.5, 4.7, 6.1, 6.2. [PML2] Ch 3.3, 3.8, 5.1, 5.2 [PML1] Ch 3.2, 5.2	Some important Classical Machine Learning Background.
Addtl. Material	Learning by Random Walks on Graphs  [notes-BHK] The Markov-chain Monte Carlo Interactive Gallery Wasserstein Gradient Flows and the Fokker - Planck Equation[notes] [not present] .Learning Dynamics with Stochastic Processes [notes]	S	Important Topics on Bayesian and Reimannian Manifold Optimization and Reinforcement Learning.

 

Project FAQ

1. How long should the project report be?

Answer: See directions in the Project section in assignments. For full points, please address each of the evaluation questions as succinctly as possible. You will get feedback on your presentations, which should also be incorporated into your final report.

Assignments, Exam, Final Project, and Presentation

There will be four take-home bi-weekly assignments, one in-class midterm exam, one take-home final project (in lieu of a final exam), and one presentation based on your project progress. The important deadline dates are:

• Midterm: Wednesday, March 26, 2:00 pm - 3:30 pm.
  
• Final Project Written Report Part 1: Due April 21st, 11:59 pm.
• Final Project Written Report, and Presentation Video, Due May 3rd, 11:59 pm

Assignments

There will be four written take-home HW assignments and one take-home final project report. Please refer to the above schedule for assignments and the final project report due time.

Assignment solutions that are turned in late shall suffer a 10% per day reduction in credit and a 100% reduction once solutions are posted.

Course Requirements and Grading

Grades will be based on these factors:

• In-class attendance and participation (5%)
• HW assignments (50% and with the potential to get extra credit) 

4 assignments. Some assignments may have extra questions for extra points you can earn. (They will be specified in the assignment sheet each time.)

• In-class midterm exam (15%) 
• First Presentation & Report (10%)
• Final Presentation & Report (20%)  

Students with Disabilities. Students with disabilities may request appropriate academic accommodations from the Division of Diversity and Community Engagement, Services for Students with Disabilities, 471-6259, http://www.utexas.edu/diversity/ddce/ssd . 

 

Accommodations for Religious Holidays. By UT Austin policy, you must notify the instructor of your pending absence at least fourteen days prior to the date of observance of a religious holiday. If you must miss a class or an examination in order to observe a religious holiday, you will be given an opportunity to complete the missed work within a reasonable time before or after the absence, provided proper notification is given.

 

Statement on Scholastic Dishonesty. Anyone who violates the rules for the HW assignments or who cheats on in-class tests or the final exam is in danger of receiving an F for the course. Additional penalties may be levied by the Computer Science department, CSEM, and the University. See http://www.cs.utexas.edu/academics/conduct