Machine Learning
Semester: |
WS 2017 |
Type: |
Lecture |
Lecturer: |
|
Credits: |
V3 + Ü1 (6 ECTS credits) |
Find a list of current courses on the Teaching page.
Type |
Date |
Room |
|---|---|---|
| Lecture/Exercise | Mon, 10:15-11:45 | UMIC 025 |
| Lecture/Exercise | Thu, 14:15-15:45 | AH I |
Lecture Description
The goal of Machine Learning is to develop techniques that enable a machine to "learn" how to perform certain tasks from experience.
The important part here is the learning from experience. That is, we do not try to encode the knowledge ourselves, but the machine should learn it itself from training data. The tools for this are statistical learning and probabilistic inference techniques. Such techniques are used in many real-world applications. This lecture will teach the fundamental machine learning know-how that underlies such capabilities. In addition, we show current research developments and how they are applied to solve real-world tasks.
Example questions that could be addressed with the techniques from the lecture include
- Is this email important or spam?
- What is the likelihood that this credit card transaction is fraudulent?
- Does this image contain a face?
Exercises
The class is accompanied by exercises that will allow you to collect hands-on experience with the algorithms introduced in the lecture.
There will be both pen&paper exercises and practical programming exercises based on Matlab (roughly 1 exercise sheet every 2 weeks). Please submit your hand written solutions of the exercises the day before the exercise class in the submission box at room 129, UMIC. Please submit your source code solutions through the L2P system.
We ask you to work in teams of 2-3 students.
Literature
For most part, the lecture will follow the book by Bishop. Additional topics are covered in Duda & Hart's book Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006 R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, 2nd Edition, Wiley-Interscience, 2000 Wherever research papers are necessary for a deeper understanding, we will make them available on this web page.
Additional Resources
- C.E. Rasmussen, C.K.I. Williams Gaussian Processes for Machine Learning MIT Press, 2006 available available online.
- D.J. MacKay Information Theory, Inference, and Learning Algorithms Cambridge University Press, 2003 also available online.
- Lecture videos provided by the Fachschaft
Matlab Resources
- Matlab Online Reference Documentation
- Getting started with Matlab
- Techniques for improving performance
- A useful Matlab Quick-reference card (in German).
| Date | Title | Content | Material |
|---|---|---|---|
| Introduction | Introduction, Probability Theory, Bayes Decision Theory, Minimizing Expected Loss | ||
| Prob. Density Estimation I | Parametric Methods, Gaussian Distribution, Maximum Likelihood | ||
| Prob. Density Estimation II | Bayesian Learning, Nonparametric Methods, Histograms, Kernel Density Estimation | ||
| Prob. Density Estimation III | Mixture of Gaussians, k-Means Clustering, EM-Clustering, EM Algorithm | ||
| Linear Discriminant Functions I | Linear Discriminant Functions, Least-squares Classification, Generalized Linear Models | ||
| Exercise 1 | Matlab Tutorial, Probability Density Estimation, GMM, EM | ||
| Linear Discriminant Functions II | Logistic Regression, Iteratively Reweighted Least Squares, Softmax Regression, Error Function Analysis | ||
| Linear SVMs | Linear SVMs, Soft-margin classifiers, nonlinear basis functions | ||
| Non-Linear SVMs | Soft-margin classifiers, nonlinear basis functions, Kernel trick, Mercer's condition, Nonlinear SVMs | ||
| Model Combination | Model Combination, AdaBoost, Exponential error, Sequential Additive Minimization | ||
| Exercise 2 | Linear Discriminants, SVMs | ||
| Adaboost | AdaBoost, Exponential error, Sequential Additive Minimization | ||
| Random Forests | Randomized Decision Trees, Random Forests, Random Ferns | ||
| Exercise 3 | AdaBoost, Random Forests | ||
| Neural Networks I | Single-Layer Perceptron, Multi-Layer Perceptron, Mapping to Linear Discriminants, Error Functions, Regularization, Multi-layer Networks, Chain rule, Gradient Descent | ||
| Neural Networks II | Backpropagation, Computational Graphs, Stochastic Gradient Descent, Minibatch Learning, Optimizers (Momentum, RMS-Prop, AdaGrad, Adam) | ||
| Tricks of the Trade | Initialization (Glorot, He), Nonlinearities, Drop-out, Batch Normalization, Learning Rate Schedules | ||
| Convolutional Neural Networks I | CNNs, Convolutional Layers, Pooling Layers, LeNet | ||
| Exercise 4 | Softmax, Backpropagation | ||
| Convolutional Neural Networks II | ImageNet Challenge, Notable Architectures, AlexNet, VGGNet, Inception, Visualizing CNNs | ||
| Tutorial | Hands-on tutorial on TensorFlow | ||
| CNN Architectures & Applications I | Residual Networks, Fully Convolutional Networks, Applications of CNNs | ||
| Word Embeddings | Dealing with Discrete Data, Word Embeddings, word2vec, GloVe, Hierarchical Softmax, Motivation for Recurrent Networks | ||
| Recurrent Neural Networks I | Plain RNNs, Backpropagation through Time, Practical Issues, Initialization | ||
| Exercise 5 | Convolutional Neural Networks | ||
| Recurrent Neural Networks II | LSTM, GRU, Applications of RNNs | ||
| Wrapping Up | Current Research Directions in Deep Learning | ||
| Repetition | Repetition | ||
| Exercise 6 | RNNs |
