Content

• Basic concepts: different architectures, learning rules, supervised and unsupervised learning. Shallow versus deep architectures. Applications in character recognition, image processing, diagnostics, associative memories, time-series prediction, modelling and control.
• Single- and multilayer feedforward networks and backpropagation, on-line learning, perceptron learning
• Training, validation and test set, generalization, overfitting, early stopping, regularization, double descent phenomenon
• Fast learning algorithms and optimization: Newton method, Gauss-Newton, Levenberg-Marquardt, conjugate gradient, adam
• Bayesian learning
• Associative memories, Hopfield networks, recurrent neural networks
• Unsupervised learning: principal component analysis, Oja's rule, nonlinear pca analysis, vector quantization, self-organizing maps
• Neural networks for time-series prediction, system identification and control; basics of LSTM; basics of deep reinforcement learning
• Basic principles of support vector machines and kernel methods, and its connection to neural networks
• Deep learning: stacked autoencoders, convolutional neural networks, residual networks
• Deep generative models: restricted Boltzmann machines, deep Boltzmann machines, generative adversarial networks, variational autoencoders, normalizing flow, diffusion models
• Normalization, attention, transformers

Course material

Study cost: 1-10 euros (The information about the study costs as stated here gives an indication and only represents the costs for purchasing new materials. There might be some electronic or second-hand copies available as well. You can use LIMO to check whether the textbook is available in the library. Any potential printing costs and optional course material are not included in this price.)

Content

4 computer exercise sessions

Course material

• Toledo.