M 8/31

Machine Learning Foundations

Overview, applications, settings

Bishop 1

W 9/2


Probability, stats

You do NOT need to read all
of these. However, you should
be familiar with the material.


Bishop Chapter 2

Bishop Appendix B

Andrew Moore Tutorial

Tom Minka's nuances of probability (advanced)

Wolfram Probability and stats

Linear algebra review:

Jing Xiang's review (CMU)

Zico Colter's review (Stanford)

M 9/7

No Class

Labor day

W 9/9

Decision Trees

Construction, pruining, over-fitting

Nilsson chapter 6

Supervised Learning: Linear Methods

M 9/14


Least squares and regression

Bishop 3

W 9/16


Logistic Regression

Bishop 4

M 9/21

Naive Bayes

Ng and Jordan, 2001

W 9/23

Online Learning for Structured Prediction

Perceptron, multi-class, structured

Blum. On-Line Algorithms in Machine Learning. 1996

Discriminative Training Methods for Hidden Markov Models:
Theory and Experiments with Perceptron Algorithms. EMNLP, 2002.

Bishop 4.1.2

Reducing Multiclass to Binary (Sections 1, 2, 3)

Supervised Learning: Non-Linear Methods

M 9/28

Support Vector Machines

Max-margin classification and optimization

Bishop 7.1

Chris Burges SVM Tutorial

Bishop Appendix E

W 9/30

Kernel Methods

Dual optmization, kernel trick

Bishop 6.1, 6.2

M 10/5

Instance based learning


Bishop 2.5

Mitchell 8-8.4

W 10/7

Ensemble Methods


Bishop 14.1,14.2,14.3

A Short Introduction to Boosting

M 10/12

Deep Networks 1

Neural Network models

Bishop 5.1,5.2,5.3,5.5

W 10/14

Deep Networks 2

Deep learning

One of the Following:

Deep Learning for NLP (without Magic)

ICML Deep Learning Tutorial (LeCun and Ranzato)

Geoffrey Hinton Tutorial (Video)

Tutorial on Deep Learning and Applications (Lee)

Unsupervised Learning

M 10/19

Midterm Exam

W 10/21


Expectation-Maximization and k-means

Bishop 9

M 10/26

EM and Clustering 1

Gaussian mixture models

Bishop 9

W 10/28

EM and Clustering 2

The EM Algorithm

Bishop 9

Graphical Models

M 11/2

Graphical models 1

Bayesian networks and conditional independence

Bishop 8.1, 8.2

W 11/4

Graphical models 2

MRFs and Exact inference

Bishop 8.3, 8.4

M 11/9

Graphical models 3


Bishop 8.3, 8.4

W 11/11

Sequential graphical models 1

Max Sum and Max Product

Bishop 13.1,13.2

Other Topics

M 11/16

Causal models, causal effects identification, do-calculus

Causal Inference (Introduction)

Judea Pearl's keynote for UAI-2012

Chapters 1 and 6 in Hernan Robins book

W 11/18

Causal Effects, estimation, g-formula, inverse weights, learning structure from data

Causal Inference (From Data to Conclusions)

Chapters 11, 12 and 13 in Hernan Robins book

Thomas Richardson's structure learning tutorial

M 11/23

No class

Thanksgiving Break

M 11/25

No class

Thanksgiving Break

M 11/30

Advanced topics

W 12/02

Dimensionality reduction

PCA, probabilistic PCA, LDA

Bishop 12.1,12.2,12.3

Max Welling's PCA Tutorial

Jonathon Shlens's PCA Tutorial (more mathematically oriented)

Lindsay Smith Tutorial (good at the basics)