CMSC 35420

Statistical Methods in AI (TTIC 100)

Prerequisites:

Catalog Description: This course gives a survey of mathematical methods in statistical modeling, inference, and learning with an emphasis on techniques widely used in speech recognition, computational linguistics and vision. Each lecture is organized around a theorem, algorithm, or technical concept. This course is appropriate either for students who wish acquire technical breadth outside of their speciality or for students who want to sample a variety of areas before deciding on an area of specialty.

Long Description: This course gives a survey of mathematical methods in statistical modeling, inference, and learning with an emphasis on techniques widely used in speech recognition, computational linguistics and vision. Each lecture is organized around a theorem, algorithm, or technical concept. This course is appropriate either for students who wish acquire technical breadth outside of their speciality or for students who want to sample a variety of areas before deciding on an area of specialty.

Part I: Information Theory, Modeling, Perception and Inference.

Lecture 1: Shannon vs. Chomsky. Entropy vs. Languages. Lecture 2: The Channel Capacity Theorem. Lecture 3: The Information Bottleneck. Lecture 4: Hidden Markov Models. The Viterbi Algorithm. Lecture 5: Probabilistic Context Free Grammars (PCFGs). Chart Parsing. Lecture 6: A* Parsing. Lecture 7: A* model matching in vision. Lecture 8: The Junction Tree Algorithm for Markov Random Fields. Lecture 9: Case Factor Diagrams. Lecture 10: Loopy Belief Propagation. Lecture 11: Min-Cut Clustering and Segmentation. Lecture 12: Alpha Expansion in Vision. Lecture 13: Probabilistic Relational Models. Lecture 14: Probabilistic Models for Natural Language Semantics.

Part II: Learning

Lecture 15: Bayesian Learning. Conjugate Priors. Lecture 16: Least Squares Regression. Lecture 17: Principle Component Analysis. Lecture 18: The PAC-Bayesian Theorem. Lecture 19: VC Dimension. Lecture 20: Boosting. Lecture 21: SVMs. Lecture 22: Logistic Regression. Lecture 23: Kernels. Lecture 24: Good-Turing. Lecture 25: On-line learning. Lecture 26: General EM. Lecture 27: MAP EM, Structural EM, Leave-One-Out EM. Lecture 28: Manifold and Invariant Learning. Lecture 29: Shannon vs. Chomsky Revisited

Instructors: David McAllester
Quarter offered: SPR
Last Verified by Sharon Salveter on 1 December, 2004.