Content

- Introduction and motivation
- Basics of statistical decision theory and pattern recognition
- Basics of convex optimisation theory, Karush-Kuhn-Tucker conditions, primal and dual problems
- Maximal margin classifier, linear SVM classifiers, separable and non-separable case
- Kernel trick and Mercer theorem, nonlinear SVM classifiers, choice of the kernel function, special kernels suitable for textmining
- Applications: classification of microarray data in bioinformatics, classification problems in biomedicine
- VC theory and structural risk minimisation, generalisation error versus empirical risk, estimating the VC dimension of SVM classifiers, optimal tuning of SVMs
- SVMs for nonlinear function estimation
- Least squares support vector machines, issues of sparseness and robustness, Bayesian framework, probabilistic interpretations, automatic relevance determination and input selection, links with Gaussian processes and regularisation networks, function estimation in RKHS.
- Applications: time-series prediction, finance  
- Kernel versions of classical pattern recognition algorithms, kernel Fisher discriminant analysis
- Kernel trick in unsupervised learning: kernel based clustering, SVM and kernel based density estimation, kernel principal component analysis, kernel canonical correlation analysis
- Applications: datamining, bioinformatics
- Methods for large scale data sets, approximation to the feature map (Nystrom method, Random Fourier features), estimation in the primal
- SVM extensions to recurrent models and control; Kernel spectral clustering; Deep learning and kernel machines; attention and transformers from a kernel machines perspective.

(10 lectures (2 hours) + 3 computer exercise sessions)

Course material

Study cost: 1-10 euros (The information about the study costs as stated here gives an indication and only represents the costs for purchasing new materials. There might be some electronic or second-hand copies available as well. You can use LIMO to check whether the textbook is available in the library. Any potential printing costs and optional course material are not included in this price.)

The course material is largely based on the textbook
J.A.K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, J. Vandewalle,  Least Squares Support Vector Machines, World Scientific, Singapore, 2002 (ISBN 981-238-151-1)
Related books:
Cristianini N., Shawe-Taylor J., An introduction to support vector machines,  Cambridge University Press, 2000.
Schoelkopf B., Burges C., Smola A., Advances in Kernel Methods: Support Vector Learning,  MIT Press, Cambridge, 1998.
Schoelkopf B., Smola A., Learning with Kernels,  MIT Press, Cambridge, 2002
Vapnik V., Statistical learning theory, John Wiley, New-York, 1998.

Is also included in other courses

• H02D3B : Support Vector Machines: Methods and Applications

Format: more information

3 computer exercise sessions

Is also included in other courses

• H02D3B : Support Vector Machines: Methods and Applications