Content

The course consists of four parts.

Part 1: Introduction: basic terminology and concepts; nearest neighbor approaches, evaluation

Part 2: Learning based on discrete search spaces

• decision trees: basic algorithm, heuristics, pruning strategies, missing value handling, multi-target trees
• rule learning and inductive logic programming: search in a refinement lattice, subsumption, least general generalization
• ensemble methods: boosting, bagging, random forests, bias-variance tradeoff in ensembles
• automata, learning theory: algorithms for learning automata, sample complexity, VC-dimension, Rademacher complexity

Part 3: Learning based on numerical optimization and search in continuous spaces

• Concepts of statistical learning: learning as optimization, loss functions, regularization
• Artificial neural networks: basic principles, specific network structures, convolutional neural nets, recurrent neural nets, auto-encoders, generative adversarial networks
• support vector machines: basic principles, SMO algorithm, kernels, sequence / tree / graph kernels
• dimensionality reduction
• methods based on matrix factorization

Part 4: Probabilistic models

• overview of PGMs
• inference and learning in these models
• statistical relational learning: Problog, Markov logic, ...
• Reinforcement learning: state-action spaces, problem setting, Q-learning, deep reinforcement learning

Course material

Slides, reader with book chapters and research articles. 

Sources that may be used for the reader include:

• Elements of statistical learning (Hastie, Tibshirani, Friedman)
• Machine Learning and Pattern Recognition (Bishop)
• Machine Learning (Flach)
• Deep Learning (Goodfellow, Bengio, Courville)
• Machine learning: a probabilistic perspective (Murphy)
• Introduction to Statistical Learning (James, Witten, Hastie, Tibshirani)

Content

• Pen-and-paper exercises, mostly oriented at gaining a more thorough understanding of algorithmic and mathematical elements of machine learning.
• Some practical exercises with machine learning tools and libraries (Weka, Scikit-learn, TensorFlow, ...).

Course material

A selection of recommended exercises, partially from the reader, will be made available to students via Toledo.  Example solutions will be provided for some exercises.

Format: more information

"flipped classroom" exercises sessions