Readings are primarily from the course textbook Machine Learning: a Probabilistic Perspective by Kevin Murphy. Readings from the textbook will be prefixed by "M". For example, M1 means Murphy chapter 1. Unless otherwise notes, skip sections that have a * in the title. These are optional.

For those who prefer Chris Bishop's Pattern Recognition and Machine Learning, I will list the corresponding readings from this textbook. These readings will be prefixed by "B". Optional readings will be written in italics.

Date

Topics

Readings

Supervised Learning: Classifiers

W 9/7

Machine Learning Foundations with Linear Regression

Introduces key concepts with linear regression

M1 PDF
B1
If you need to review relevant math, do it now.

M 9/12

Linear Regression (Continued)

M7, excluding M7.4 and M7.6
B3
Those without a probability background should read M2.1-2.4.1
Other probability background readings: B2, B Appendix B, Andrew Moore Tutorial, Tom Minka's nuances of probability

W 9/14

Logistic Regression

Introduces classification methods

M8 (stop at M8.3.7), M13.3
B4

M 9/19

Perceptron

Online learning algorithms

M8.5

W 9/21

Support Vector Machines

Max-margin classification and optimization

M14.5
Andrew Ng's notes on SVMs
B7.1

M 9/26

Kernel Methods

Dual optimization, kernel trick

M14.1, M14.2
B6.1, B6.2

W 9/28

Decision Trees

Construction, pruning, over-fitting

M2.8, M16.2
Tom Mitchell's notes on decision trees

M 10/3

Catchup day



W 10/5

Boosting

Ensemble methods

M16.4, M16.6
A Short Introduction to Boosting
B14.1, B14.2, B14.3

M 10/10

Deep Learning 1

M16.5, M27.7, M28
B5.1, 5.2, 5.3, 5.5

W 10/12

Deep Learning 2

M 10/17

Midterm

Unsupervised Learning: Core Methods

W 10/19

Probability

M2

Th 10/20

Clustering

K-means

M25.1, M11 (stop at M11.4)
B9

M 10/24

Expectation Maximization 1

M11.4 (stop at M11.4.8)
Andrew Ng's notes on EM

W 10/26

Expectation Maximization 2

K-means

M 10/31

Graphical Models 1

Bayesian networks and conditional independence

M10
B8.1, B8.2

W 11/2

Graphical Models 2

MRFs and exact inference

M19.1 (stop at M19.4), M19.5
B8.3, B8.4

M 11/7

Graphical Models 3

Inference

M20 (stop at M20.3)

W 11/9

Graphical Models 4

Max Sum and Max Product

M 11/14

Structured Prediction 1

Margin based methods, HMMs, CRFs

M17 (stop at M17.6), M19.6, M19.7
Sutton, McCallum CRF tutorial
B13.1, 13.2

W 11/16

Structured Prediction 2

Recurrent Neural Networks

M 11/28

Dimensionality reduction

PCA

M12.2
B12.1, B12.2, B12.3

W 11/30

Nanyun Peng: Deep model + graphical models

M 12/5

Ilya Shpitser: Causal Inference

W 12/7

Practical Machine Learning

F 12/18

NO FINAL EXAM