Data Mining and Neural Networks

All programmes > Data Mining and Neural Networks

Data Mining and Neural Networks (B-KUL-H03V7B)

4 ECTS

English

First term

Suykens Johan

POC Wiskundige ingenieurstechnieken

Aims

Content:

Many application areas are characterized by a growing number of data, which are available and should be explored for improved modelling, efficient and automatic processing of data and extracting knowledge from the data. Typical examples include pattern recognition, biomedical engineering and bioinformatics, signal processing and system identification, process industry, fraud detection, web mining, e-commerce, financial applications, etc. In each of these areas, artificial neural networks are an important technique for analysis and design of systems. Neural networks are universal approximators, possess a parallel architecture and learn on-line or in batch mode from given sample patterns and lead to powerful methods for modeling. Training of neural networks can be done either supervised or unsupervised.

This course provides an overview of the main classical and advanced modern techniques on data mining and neural networks. Commonly used types of neural networks (such as multilayer perceptrons, radial basis function networks) are discussed, including structure, learning algorithms, optimization methods, on-line versus batch training, generalization aspects, validation, feedforward and recurrent networks, statistical interpretations, pruning, variance reduction, decision functions, density estimation and regularization theory. Special attention is given to efficient and reliable algorithms for classification and function estimation and processing of large data sets for data mining applications. Furthermore, emphasis is given on preprocessing, feature selection, dimensionality reduction and incorporation of expert knowledge. In addition to the classical neural network techniques in supervised learning more advanced methods are also addressed such as Bayesian inference, deep learning, statistical learning theory and support vector machines. With respect to unsupervised learning, cluster algorithms (and related methods such as EM algorithm), vector-quantization and self-organizing maps are discussed. Starting from linear and nonlinear principal component analysis, principles of stacked autoencoders and convolutional neural networks are explained for deep learning. Furthermore, deep learning based on attention and transformers, and generative models are discussed.

Lectures:

1. Introduction
2. Multilayer feedforward networks and backpropagation
3. Nonlinear modelling and time-series prediction
4. Classification and Bayesian decision theory
5. Generalization, Bayesian learning of neural networks
6. Vector quantization, self-organizing maps, regularization theory
7. Basic principles of support vector machines and kernel-based models
8. Nonlinear principal component analysis, autoencoders, deep learning with stacked autoencoders and convolutional neural networks
9. Generative models: deep Boltzmann machines, generative adversarial networks, variational autoencoders, others
10. Normalization, attention, transformers