Content

Extensions for multi-variate data.

- Cluster analysis

- Principal Component Analysis

Course material

Course notes, weblectures and R scripts are available at Toledo.

Content

Extensions for multi-variate data.

Course material

Course notes and R scripts are available at Toledo.

Content

Chapter 1: Descriptive statistics

Chapter 2: Important distributions

• Discrete distributions
• Continuous distributions

Chapter 3: Confidence Interval

Chapter 4: Hypothesis testing

• Concepts
• Z test
• One sample t test
• Two sample t test with equal/unequal variances
• F test for equal variances
• Binomial test
• Chi-squared test
• Normality test
• Wilcoxon test
• Power analysis

Chapter 5: Linear regression

• Correlation coefficient
• Simple linear regression (least squares method, statistical inference, diagnostics, influential observations)
• Multiple regression (regression model, diagnostics, influential observations)
• Polynomial regression
• Interaction
• Qualitative predictor variables

Chapter 6: Analysis of Variance

• One-way Anova (F test, assumptions)
• Multiple testing (Bonferroni, Tukey)
• Two-way Anova

Chapter 7: General linear model

Chapter 8: Introduction to logistic regression

• Simple logistic regression
• Multiple logistic regression
• ROC curve

Chapter 9: Introduction to Poisson regression

Chapter 10 : Introduction to Generalised linear model

Course material

Course notes, web lectures and R scripts are available on Toledo;

Format: more information

There are web lectures available. Q&A sessions will be organised. 

Is also included in other courses

I0U35A : Univariate Data and Modelling

Content

Exercises with R software are completed on the topics dealt with in Univariate Data and Modelling, part I.

Course material

R scripts, data sets and exercise material are available on Toledo.

Format: more information

Exercises are done in a pc class with the R software.

Is also included in other courses

I0U35A : Univariate Data and Modelling

Explanation

The exam is an open book exam and takes place in a pc class where the students can use R.

Content

The course covers the following topics:

• Introduction to computer privacy and privacy engineering
• Database anonymity and privacy: k-anonymity, l-diversity, t-closeness, re-identification attacks, statistical disclosure control, differential privacy
• Privacy by design
• Privacy and machine learning
• Privacy risk analysis
• Web privacy and user tracking
• General Data Protection Regulation (GDPR), human rights legislation, relevant policy for Big Data, data protection impact assessments
• Ethical issues of Big Data, Discrimination-aware data-mining, algorithmic accountability

Course material

Slides, courseware, articles and literature

Is also included in other courses

H00Y2B : Privacy and Big Data

Content

The first two practical sessions will be devoted to group discussions and feedback on drafts of the assignment. Students work together on the assignment in teams of 2 or 3 people (see evaluation for more details). Students working together on the same team should be in different groups during the exercise session discussions, in order to maximize the feedback obtained from students in other groups. 

In the first session students will discuss their initial ideas for the assignment (description of their chosen application and initial identification of privacy issues).

In the second session students will discuss a more complete draft of their assignment and get a second round of feedback for further improvement.

The last two practical sessions will be devoted to presentations of the assignments by the students. The presentations will be graded (4 points out of 20). Students get feedback on their presentation that they can incorporate in the final version of the assignment. 

Course material

Slides, courseware, articles and literature

Is also included in other courses

H00Y2B : Privacy and Big Data

Explanation

Students will select (in teams of 2 or 3 people) a case study in the second week of the class. The case study is an application of service that utilizes Big Data.

Students must perform a privacy impact assessment, meaning in-depth analysis of the case study with respect to the privacy (legal, technical, and ethical) aspects covered in the class, in an assignment of between 3500 and 4500 words (+/- 15 pages). The assignment text includes: description of the application, analysis of privacy issues and proposed recommendations to address those issues. 

The teams present their work in a short presentation (5-10 minutes, depending on number of teams) followed by some questions and feedback. The text of the assignment is due after the presentation sessions and before the start of the examination period. 

The presentation is graded with 4 points and the final text of the assignment with 16 points. 

Information about retaking exams

In the second examination period all 20 points are evaluated on the basis of the written assignment (no presentation). Assignments must be submitted BEFORE the start of the examination period (deadline is the last day before the examnination period starts). 

Content

Note that the order that the topics are covered in can vary from year to year.

Part I: Basics
1. Introduction and overview
2. Background on hashing, computer organization, complexity basics, etc.
3. Databases basics: SQL, join algorithms, index structures
4. Advanced topics: Fancy indexes, column store, warehouses, noSQL
 

Part II: Structures and techniques for efficiency
1. Introduction an overview
2. Learning from data streams
3. Fast nearest neighbors algorithms
4. Implementation tricks
5. Approximation methods (e.g., sketches, sampling)
6. Advanced topics?

Part III: Parallel Architectures
1. Introduction and overview
2. Types of parallelism (e.g., shared memory, shared nothing)
3. Concurrency
4. Parallel programming bugs (e.g., data races, deadlock, etc.)
5. Map-reduce
6. Cloud computing
7. Condor?

Course material

Lecture slides, readings, and online resources

Content

1. Part I: Query Languages
1. Introduction and overview
2. SQL: selection, projection, select-project-join, group-by, aggregates, subqueries, nested queries
3. Xquery
4. Sparql
5. Writing applications that can interface with a DBMS
6. Indexing?

2. Part II: Scripting Languages
1. Introduction an overview
2. Perl/Python

3. Part III: Parallel Architectures
1. Introduction and overview
2. Types of parallelism (e.g., shared memory, shared nothing)
3. Concurrency
4. Parallel programming bugs (e.g., data races, deadlock, etc.)
5. Map-reduce
6. Cloud computing
7. Condor?

Course material

Exercise slides

Content

Examples of the types of assignments
1. Given a set of verbal queries, translate them into a query language
2. Write queries to extract information needed for a machine learning task
3. Implement advanced machine learning algorithms (e.g., for learning from streaming data)
4. Implement an advanced data mining algorithm
5. A project that uses Hadoop, Spark, etc.

Course material

Assignment sheets

Explanation

The evaluation of the course will be based on multiple  programming assignments. Solutions are evaluated in terms of correctness, efficiency and generalizability.

Projects that are independent mean that students must complete the assignment individually. Thus using outside sources (e.g., publicly available code, etc.) or working together (e.g., working with somone else to solve the assignment, getting substantial help from someone else to solve the assignment, etc.) is strictly forbidden. If you questions about what is and is not permitted, please consult the instructor.

Information about retaking exams

For the project assignments with a failed result, the student will have an opportunity to complete an alternative assignment.

Content

Students take active part in research related questions within one of the sub-domains of A.I. In particular, students select one of the research projects offered by the involved research units. They work independently, but guided by the staff of the research unit, to perform the research work required for the selected project.
In the BD-option, the thesis-project should involve the development of or the experimentation with a small-scale Big Data related system. Part of the master's thesis should consist of a critical positioning of the topic of the thesis-project within the broader setting of A.I.

In addition to the possibility of developing and writing a master thesis, the students also have the possibility to do a company project of 15 weeks or a project within a research institute, either in Belgium or abroad. The main activities of the internship are:
- prepare a detailed job description for the internship; this is done in collaboration with a staff member of the host institution (the supervisor) and has to be endorsed by a staff member of the MAI Faculty (the academic promoter).
- carry out the project which is specified in the job description under the supervision of a staff member of the host institution; send monthly progress reports.
- write a report which summarizes the results of the project work; it should have at least 20 pages (annexes not included).
- oral presentation and defense of the report.

Course material

Articles and literature

Explanation

The thesis project should take the form of a publishable scientific paper.  The thesis project must be defended in an oral presentation, before a jury consisting of the thesis advisor and two readers.

Content

1. Introduction
- Definition and general context, both of the domain and the course
 
2. State-space representation and search methods
- state-space representation: introduction and trade-offs,
- blind search,
- heuristic search, including the study of the A*-algorithm,
- advanced aspects of heuristic search,
3. Examples of search
- search in data mining: pattern mining
- heuristic search in games
4. Constraint propagation
- backtracking, backtrack variants, intelligent and dependency directed backtracking,
- arc consistency techniques,
- hybrid constraint propagation methods
 5. Some case studies of the use of constraint processing
- interpretation of line drawings,
- interpretation of natural language
 6. Planning and Temporal representation
- partial-order regression planning: STRIPS

Course material

Copies of the slides are made available by the student organisation VTK.

Is also included in other courses

H02A0C : Fundamentals of Artificial Intelligence

Content

1. Introduction (2 u.)
- Definition and general context, both of the domain and the course

2. State-space representation and search methods (8 u.)
- state-space representation: introduction and trade-offs,
- blind search,
- heuristic search, including the study of the A*-algorithm,
- advanced aspects of heuristic search,
- heuristic search in games
3. Examples of search: pattern mining and games (2 u.)
4. Constraint propagation (3 u.)
- backtracking, backtrack variants, intelligent and dependency directed backtracking,
- arc consistency techniques,
- hybrid constraint propagation methods
5. Some case studies of the use of constraint processing (3 u.)
- interpretation of line drawings,
- interpretatie of natural language
6. Planning and Temporal representation (1.5 u.)
- partial-order regression planning: STRIPS

Course material

Slides of the study material are available from the student organisation VTK

Is also included in other courses

H02A0C : Fundamentals of Artificial Intelligence

Content

The projects challenge the student to implement AI search techniques seen in class, in a Python programming environment based on the PacMan world developed at UC Berkeley. 

The project consists of two parts: 

• P1: Search: Students implement depth-first, breadth-first, uniform cost, and A* search algorithms. These algorithms are used to solve navigation and traveling salesman problems in the Pacman world. 
• P2: Multi-Agent Search: Classic Pacman is modeled as both an adversarial and a stochastic search problem. Students implement multiagent minimax and expectimax algorithms, as well as designing evaluation functions. 

You will fill in portions of provided Python files during the assignment. An autograder is provided to grade your own answers and guide you in your development; your submission will be evaluated by us 
independently. Collaboration and plagiarism are not allowed, we will check for this extensively and sanction if needed. 

Course material

The project assignment and the necessary software libraries and installation instructions will be made available on Toledo.

Explanation

Your final grade is determined by three parts: the project, the lecture exam and the exercise exam. The exact point distribution will be communicated through the online learning platform.

The project(s) will be evaluated during the year. They have to be made individually and plagiarism will be sanctioned. 

The exam is a written exam consisting of multiple-choice questions, fill-in questions and open questions. The exam consists of two parts: the part on the lecture material will test for factual and synthetic knowledge; the part on the exercises will test your ability to solve exercises involving the lecture material, as practiced in the exercise sessions. 

Content

1. Introduction
- Definition and general context, both of the domain and the course
 
2. State-space representation and search methods
- state-space representation: introduction and trade-offs,
- blind search,
- heuristic search, including the study of the A*-algorithm,
- advanced aspects of heuristic search,
3. Examples of search
- search in data mining: pattern mining
- heuristic search in games
4. Constraint propagation
- backtracking, backtrack variants, intelligent and dependency directed backtracking,
- arc consistency techniques,
- hybrid constraint propagation methods
 5. Some case studies of the use of constraint processing
- interpretation of line drawings,
- interpretation of natural language
 6. Planning and Temporal representation
- partial-order regression planning: STRIPS

Course material

Copies of the slides are made available by the student organisation VTK.

Is also included in other courses

H02A0A : Fundamentals of Artificial Intelligence

Content

1. Introduction (2 u.)
- Definition and general context, both of the domain and the course

2. State-space representation and search methods (8 u.)
- state-space representation: introduction and trade-offs,
- blind search,
- heuristic search, including the study of the A*-algorithm,
- advanced aspects of heuristic search,
- heuristic search in games
3. Examples of search: pattern mining and games (2 u.)
4. Constraint propagation (3 u.)
- backtracking, backtrack variants, intelligent and dependency directed backtracking,
- arc consistency techniques,
- hybrid constraint propagation methods
5. Some case studies of the use of constraint processing (3 u.)
- interpretation of line drawings,
- interpretatie of natural language
6. Planning and Temporal representation (1.5 u.)
- partial-order regression planning: STRIPS

Course material

Slides of the study material are available from the student organisation VTK

Is also included in other courses

H02A0A : Fundamentals of Artificial Intelligence

Content

The projects challenge the student to implement AI search techniques seen in class, in a Python programming environment based on the PacMan world developed at UC Berkeley. 

The project consists of the following: 

• P1: Search: Students implement depth-first, breadth-first, uniform cost, and A* search algorithms. These algorithms are used to solve navigation and traveling salesman problems in the Pacman world. 

You will fill in portions of provided Python files during the assignment. An autograder is provided to grade your own answers and guide you in your development; your submission will be evaluated by us independently. Collaboration and plagiarism are not allowed, we will check for this extensively and sanction if needed. 

Course material

The project assignment and the necessary software libraries and installation instructions will be made available on Toledo. 

Explanation

Your final grade is determined by three parts: the project, the lecture exam and the exercise exam. The exact point distribution will be communicated through the online learning platform.

The project(s) will be evaluated during the year. They have to be made individually and plagiarism will be sanctioned. 

The exam is a written exam consisting of multiple-choice questions, fill-in questions and open questions. The exam consists of two parts: the part on the lecture material will test for factual and synthetic knowledge; the part on the exercises will test your ability to solve exercises involving the lecture material, as practiced in the exercise sessions. 

Content

Without being exhaustive, typical paradigms that could be presented are:

• constraint programming
• planning languages and systems
• theorem proving and systems for formal verification
• probabilistic reasoning and neuro-symbolic languages
• AI-extensions of logic, functional or object-oriented programming languages

For each introduced language, system or methodology, the important themes are the conceptual foundations, the main built-in representation and problem solving features, the illustrations of use and limitations, the indication of the key research aspects and the key applications.

The selection of studied paradigms can vary from year to year.

Course material

Study cost: 76-100 euros (The information about the study costs as stated here gives an indication and only represents the costs for purchasing new materials. There might be some electronic or second-hand copies available as well. You can use LIMO to check whether the textbook is available in the library. Any potential printing costs and optional course material are not included in this price.)

Slides and lecture material will be made available on the learning platform.

Content

The exercise sessions are related to the material of the course.

Course material

Exercises and necessary background material will be made available on the learning platform.

Explanation

Exam questions can contain a mix of open, closed and multiple choice questions. In case of multiple choice questions a guessing correction scheme will be communicated and used. The exact format of the exam will be announced on the learning platform.

Content

Robotics – theory lectures (3 ects) 

Lectures cover: 

• introduction to software development for robots 
• robot kinematics and dynamics 
• robot motion control and sensor-guided control, free-space and in-contact tasks 
• trajectory optimization (based on numerical optimization techniques) 
• dealing with uncertainty / estimation in robotics 
• classical motion planning and learning methods for motion planning 
• learning from demonstration 

Robotics – exercise and (virtual) laboratory sessions (1 ects) 

• (Computer) exercises and interactive lab visits on the lecture material

Course material

The study material consists of lecture notes, including paper references and book excerpts. 

Explanation

Students, in groups of two or, exceptionally, individually: 

• Do a homework assignment consisting of a set of computer programming exercises, and hand in a short report and their code. 
• Do a short research project. A set of possible projects will be made available by the lecturers. 

During the examination period there is an oral defence covering both the homework assignment and the research project. The homework assignment counts for 25% of the grade, the research project for 75%. 

Content

Part I: Early and Mid-Level Vision

1.   Introduction to Computer Vision

2.   Basic Image Processing

• Linear Filtering
• Pyramids/Template Matching
• Non-linear Filtering (median, bilateral filtering)
• Morphology

3.   Feature Detection and Matching

• Edges
• Points/Patches
• Fitting
• Hough

4.   Grouping & Segmentation

• Clustering (K-Means, Agglomerative Clustering, Mean Shift)
• Spectral Clustering (Normalized Cuts, Graph cuts)
• Deformable Contours (Active Contours, Dynamic Programming)

Part II: High-level Vision

5.   Introduction to Image Understanding

6.   Object Detection

• Scanning/Sliding Window
• Eigenfaces, HOG, LBP
• Boosting (Intro, Haar-Cascade)

7.   Instance Recognition

• Local Feature Matching
• Bag of Words
• Spatial Verification

8.   Deep Learning for Image Classification

• Intro to Deep Learning
• DCNN for Image Classifcation

Course material

Slides and references to online available textbooks and papers

Content

Concepts presented in the lectures as well as complementary aspects are further discussed through hands-on experience.

Three assignments are planned. They all make use of the Python scripting language. 

• The first assigment helps you to get acquainted with basic image processing steps required for further image analysis. 
• The second assignment presents different (both traditional and DL) approaches to object detection and recognition.
• The third assignment helps the student mastering DL approaches for image classification and segmentation

Course material

Python scripting language. Online references. 

Explanation

The evaluation is based on the three assignments.

The first is an individual assignment and will be evaluated on the quality and originality of the submitted output (a video showing image processing results). 

The second and third assignment are group assignments. They will be evaluated on the quality of the submitted reports. 

The weighting of the different assignments is (20%, 40%, 40%). For the second and third assignment, peer assessment will be used to detect gross asymmetric contributions of group members. 

Content

1. Speech Signal Processing and The Auditory Scene 

• the spectrogram, spectral estimation for acoustic signals
• elementary psychoacoustic concepts (loudness, pitch, timbre, critical bands)
• source-filter model of speech production, speech features (formants)
• mel spectrogram and cepstral features, spectral distance measures
• Dynamic Time Warping for matching signals that vary in time and frequency

2. Speech Recognition

• Classification of speech sounds and speech meta data
• Hidden Markov models, Viterbi decoding, training of  HMMs
• Decision Trees for Context-Dependent Phone modeling
• Linguistic resources: dictionaries, language models
• Large vocabulary decoding
• Static Deep Neural Nets for classifying speech sounds, Hybrid systems
• Time Warping Deep Neural Networks for Speech Recognition, a.o.:
  • Long Short-Term Model (LSTM), 
  • Connectionist Temporal Classification (CTC)
• Deep Neural Nets for language modeling
• Industrial outlook, applications of Speech Recognition

Course material

Course notes are available online

https://homes.esat.kuleuven.be/~compi/H02A6/course_notes/html

Additional material is available via Toledo

Content

There is a combination of lab sessions and exercises, all details are available in TOLEDO.  There are typically 6 sessions (though small deviations from the prototypical sequence are possible)

1. The Auditory Scene, Time - Frequency  representations of signals (speech & music)

2. Feature Extraction, Spectral Distance Metrics, Dynamic Time Warping

3. HMMs, Viterbi recognition

4. HMM training, context-dependent models, hybrid systems

5. DNNs for Language Modeling

6. Time Warping DNNs: CTC, LSTM 

Format: more information

All exercises are hand on sessions working out examples on pencil and paper, or - most of the time - working with Jupyter notebooks.

Students are expected to have enough computer skills to run Jupyter notebooks (Python), to understand the flow of the code by reading the - well documented - top level scripts and to be able to make minimal changes (such as changing variable assignments or skipping pieces of code).  Having some prior exposure to any scripting language is therefore deemed sufficient as a prerequisite.

Explanation

Open book exam with several exercises covering the different topics of the course. 

If, for organizational reasons (e.g. too many registered students), it is concluded that the organization of an oral exam is not feasible, then the oral exam will not take place and the written preparation will become a written exam. The impact of this decision will be explained on Toledo.

Example exams can be found here: http://homes.esat.kuleuven.be/~spchlab/H02A6/exams

Content

1. Introduction

• What is natural language processing (NLP)?
• Current state-of-the-art of NLP
• Ambiguity
• Other challenges
• Representing words, phrases and sentences
   

2. Segmentation and tokenization

• Regular expressions
• Word tokenization, lemmatization and stemming
• Sentence segmentation
• Subword tokenization
   

3. Language Modelling

• N-gram language models
• perplexity
• maximum likelihood estimation
• smoothing
   

4. Neural Language Modelling

• Word embeddings
• Vector space models for NLP
• Recurrent neural network (RNN) for language modelling
• Transformer architecture for language modelling
• Use of language models in downstream tasks: fine-tuning and pretraining
   

5. Part-of-Speech (POS) Tagging

• Hidden Markov model and viterbi
• Conditional Random Fields
• (Bi)LSTM for POS tagging
• Encoder-decoder architecture for sequence-to-sequence labeling
   

6. Morphological analysis

• Inflection and derivation
• Finite state morphology
• Sequence-to-sequence neural models of morphological inflection
   

7. Syntactic Parsing

• Universal Dependencies
• Dependency parsing: Graph based dependency parsing, transition based dependency parsing
• Constituent parsing with a (probabilistic) context free grammar ((P)CFG) and the Cocke-Younger-Kasami (CYK) algorithm
   

8. Semantics (lexical and compositional)

• Word sense disambiguation
• Semantic role labelling
   

9. Discourse: Coreference Resolution

• Discourse coherence
• Algorithm of Hobbs
• Neural end-to-end coreference resolution
   

10. Question Answering

• Evolution of QA systems from rule-based to neural 
• Complex pipelines to end-to-end to retrieval-free
• Closed-domain vs open-domain
• Text-only vs multimodal
   

11. Neural Machine Translation

• Encoder-decoder architecture (e.g., RNN, transformer-based)
• Attention models
• Improvements and alternative architectures that deal with limited parallel training data
   

12. Conversational Dialogue Systems and Chatbots

• Task oriented dialog agents: Rule based versus neural based approaches
• Chatbots: End-to-end sequence-to-sequence neural models
   

Course material

Handbooks

Daniel Jurafsky and James H. Martin. 2024. Speech and Language Processing: An Introduction to Natural
Language Processing, Computational Linguistics, and Speech Recognition with Language Models, 3rd edition.

Jacob Eisenstein. 2019. Introduction to Natural Language Processing. MIT Press.

Yoav Goldberg. 2016. A Primer on Neural Network Models for Natural Language Processing.

Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press.

  
+ recent articles: e.g., of the proceedings of the Meetings of the ACL, AAAI, NeurIPS.

Format: more information

Interactive lectures with short exercises.

Is also included in other courses

G0D25A : Natural Language Processing

Content

• Exercises on tokenization and segmentation
• Exercises on language modelling and POS tagging
• Exercises on syntactic parsing
• Exercises on semantic and discourse processing
• Exercises on machine translation
• Exercises on question answering

Is also included in other courses

G0D25A : Natural Language Processing

Explanation

Open book written exam featuring a mixture of theory and exercise questions.

Content

The course discusses 4 main topics:
1) general discussion of what cognitive science is
2) thinking and reasoning, with extra emphasis on the infant's object concepts and on rationality
3) perception, with an extra emphasis on the brain as a hypothesis-constucting-and-testing agent and on eye-movements
4) language, with an extra emphasis on optimality theory and language acquisition

*

The lectures follows closely the chapters from Cognitive Psychology” by Ken Gilhooly, Fiona Lyddy and Frank Pollick (McGraw Hill, 2014)

Content

The course discusses 4 main topics:
1) general discussion of what cognitive science is
2) thinking and reasoning, with extra emphasis on the infant's object concepts and on rationality
3) perception, with an extra emphasis on the brain as a hypothesis-constucting-and-testing agent and on eye-movements
4) language, with an extra emphasis on optimality theory and language acquisition

Explanation

With respect to the final written exam: the first hour is a closed book exam with 21 MC-questions. This part counts for 7 points. The next two hours are open book, where you have to solve two questions. This part counts for 7 points.

- With respect to the paper: the deadline for the paper will be communicated via Toledo at the beginning of the semester. If you miss the deadline, you will receive no points. This part counts for 5 points.

- With respect to the discussion board: you will post 2 entries and respond to at least 2 entries of other students on at least 3 of the discussion forums. This part counts for 1 point.

Content

Part I: The nervous system
neuron, action potential, synapse, channel regulation and memory, visual
system, neural maps and the development of the visual system, reaching
and grasping

Part II: Recording from the brain
invasive and non-invasive recording

Part III: Computational neuroscience
principles, Reichardt detector, models of complex motion and actions,
convolutional and self-organising neural networks
 

Course material

Study cost: 1-10 euros (The information about the study costs as stated here gives an indication and only represents the costs for purchasing new materials. There might be some electronic or second-hand copies available as well. You can use LIMO to check whether the textbook is available in the library. Any potential printing costs and optional course material are not included in this price.)

course slides, course texts
course overview
example questions for the exam

all available from Toledo
 

Format: more information

Lectures with examples and case studies to promote student interaction.

Content

Lab sessions in support of the course material.

Explanation

Written exam.

Sample questions are available from the course's Toledo page.

Content

Using three different paradigms (the symbolic, the statistical, and the neural paradigm), we examine how language can be modelled computationally, and how different language applications can be developed accordingly. We discuss the underlying linguistic phenomena, and we examine how they can be modelled and automatically learned from data using statistical and neural techniques. We look at various language processing applications, such as sentiment analysis and machine translation. We also discuss the risks associated with natural language processing, and the drawbacks of current language processing models based on neural representations, such as bias and uninterpretability. During the practical sessions, we will explore hands-on how state of the art language models can be used for various language processing applications, for language generation, and for linguistic research.

Course material

Lecture slides and recommended background reading, provided through Toledo

Is also included in other courses

G0D21A : Linguistics and Artificial Intelligence

Content

Practical assignments on natural language processing models

Course material

Electronic handout in the form of Jupyter notebooks

Is also included in other courses

G0D21A : Linguistics and Artificial Intelligence

Content

1. introduction to machine learning, connections with other subjects
2. general principles of learning:
- concept learning, version spaces
- evaluation of learning algorithms
- theory of learnability
- representation of inputs and outputs of learning algorithms
3. specific learning approaches:
- decision trees
- rules, association rules
- instance based learning
- clustering
- neural networks
- support vector machines
- Bayesian learning
- genetic algorithms
- ensemble methods (bagging, boosting, ...)
- reinforcement learning
- inductive logic programming
 

Course material

Course Text
Lecture slides

Format: more information

Ten lectures of 2 hours each.
 

Content

Exercises are made on the subjects discussed during the lectures. These are mostly pen-and-paper exercises where students gain insight in the workings of learning algorithms by manually mimicking the computations of certain learning algorithms, graphically describing the result of a learning algorithm (by drawing decision surfaces), etc. There are also exercises on evaluation of machine learning models and algorithms.

Course material

• A list of exercises.
• Solutions are made available on Toledo.

Format: more information

Students try to independently solve the exercises during some time. A teaching assistant provides help where necessary, and discusses the solution afterwards.

Explanation

The exam consists of questions about the theory and exercises. A formula sheet can be consulted during the exam. 

If the evaluation shows that the student does not meet one or more objectives of the course, the global result may differ from a weighted average of the parts.

Content

• Introduction on the role of Knowledge Representation  in AI
  (Informal introduction to different types of knowledge and propositional attitudes, possible world analysis of knowledge, the role of knowledge in problem solving, the controversies and trade-offs of KR in AI)
•  Knowledge representation in classical logic
  (Syntax, informal and formal semantics of classical logic, KR methology in classical logic)
• Extending classical logic with definitions
  (introduction to different types of definitions and inductive definitions, syntax and formal semantics of ID-logic)
• Using classical logic for problem solving
  (The role of different forms of logical inference in AI and for problem solving)
•  Epistemic modal logic
  (Reasoning about knowledge, beliefs and intentions of other agents, syntax, formal and informal semantics of modal logic, correspondence theory, application to multi-agents systems)
• Knowledge representation in probabilistic logics
  (Introduction to probabilistic logics. Case study of CP-logic: syntax, informal and formal semantics.)
• Introduction to non-logical  KR-formalisms
  (Production rules and frame-based systems)

Course material

• Slides on Toledo
• Book "Knowledge Representation and Reasoning" Ronald Brachman and Hector Levesque

Content

• Introduction on the role of Knowledge Representation  in AI
  (Informal introduction to different types of knowledge and propositional attitudes, possible world analysis of knowledge, the role of knowledge in problem solving, the controversies and trade-offs of KR in AI)
•  Knowledge representation in classical logic
  (Syntax, informal and formal semantics of classical logic, KR methology in classical logic)
• Extending classical logic with definitions
  (introduction to different types of definitions and inductive definitions, syntax and formal semantics of ID-logic)
• Using classical logic for problem solving
  (The role of different forms of logical inference in AI and for problem solving)
•  Epistemic modal logic
  (Reasoning about knowledge, beliefs and intentions of other agents, syntax, formal and informal semantics of modal logic, correspondence theory, application to multi-agents systems)
• Knowledge representation in probabilistic logics
  (Introduction to probabilistic logics. Case study of CP-logic: syntax, informal and formal semantics.)
• Introduction to non-logical  KR-formalisms
  (Production rules and frame-based systems)

*

The excercise sessions address the following topics:

• KR methodology using classical logic
• Syntax , semantics and KR of definitional knowledge
• Use of inference tools for automated problem solving using declarative specifications
• Modal and multi-modal logics, Kripke structures
• Probabilistic logics, syntax and semantics of CP-logic

Course material

• Exercises and model solutions are made available on Toledo
• Students use the state-of-the-art inference system IDP to evaluate  the correctness of their solutions for KR excercises
• The system is also used to show the role of inference for automated problem solving using declarative specifications.

Explanation

Written exam (3h) :

•  Theoretical part: closed book
•  Exercise part:: open book  : only slides of the course
•  June and September
• No projects for this course

Content

• Basic concepts: different architectures, learning rules, supervised and unsupervised learning. Shallow versus deep architectures. Applications in character recognition, image processing, diagnostics, associative memories, time-series prediction, modelling and control.
• Single- and multilayer feedforward networks and backpropagation, on-line learning, perceptron learning
• Training, validation and test set, generalization, overfitting, early stopping, regularization, double descent phenomenon
• Fast learning algorithms and optimization: Newton method, Gauss-Newton, Levenberg-Marquardt, conjugate gradient, adam
• Bayesian learning
• Associative memories, Hopfield networks, recurrent neural networks
• Unsupervised learning: principal component analysis, Oja's rule, nonlinear pca analysis, vector quantization, self-organizing maps
• Neural networks for time-series prediction, system identification and control; basics of LSTM; basics of deep reinforcement learning
• Basic principles of support vector machines and kernel methods, and its connection to neural networks
• Deep learning: stacked autoencoders, convolutional neural networks, residual networks
• Deep generative models: restricted Boltzmann machines, deep Boltzmann machines, generative adversarial networks, variational autoencoders, normalizing flow, diffusion models
• Normalization, attention, transformers

Course material

Study cost: 1-10 euros (The information about the study costs as stated here gives an indication and only represents the costs for purchasing new materials. There might be some electronic or second-hand copies available as well. You can use LIMO to check whether the textbook is available in the library. Any potential printing costs and optional course material are not included in this price.)

Content

4 computer exercise sessions

Course material

• Toledo.

Explanation

Individually written report about the exercise sessions, with additional oral discussion.

Content

Topics covered include (not necessarily in this order):

1) Data mining overview

2) The data mining process

3) Recommender systems

4) Association rule mining

5) Sequential pattern mining

6) Clustering

7) Large scale decision tree learning

8) Advanced topics on ensemble methods

9) Using unlabeled data

10) Data streams

11) Advanced topics (time permitting)

Content

The exercise sessions reinforce the central concepts covered during class and give students some experience working with publicly available data mining tools.  More specifically, tasks many include:

1) Working through the control of an algorithm to better understand how it functions

2) Implementing a small part of an algorithm

3) Working through a small part of the data mining process

4) Using Weka to analyse data

5) Theoretical questions designed to extend a student's knowledge of the subject

6) Discussing and solving a data mining problem with a small group and presenting the conclusions of the discussion to the whole exercise session

Explanation

Closed book written exam about the topics covered in the lectures, exercise sessions and reading.  The goal will be to assess two questions:

1) Do you understand the important basic concepts covered in class

2) Do you have an advanced understanding of the topics covered

Some questions will be similar in spirit to those solved in the exercise sessions while others will ask a student to apply a learned concept in a different context.  Be sure to read all the questions carefully and to think about how the answer to each question is structured.  

Content

The lectures are organized into the following topics:

1. General concepts of biometrics

2. Fingerprint recognition

3. Iris recognition

4. Face recognition

5. Hand recognition

6. Signature recognition

7. Keystroke recognition

8. Retina recognition

9. Ear recognition

10. Invited Lecture on legal and ethical implication in biometrics

Course material

Slides, recordings and online accessible material

Content

The Biometrics lectures are complemented by a series of programming (Python-based) and/or literature assignments giving the student hands-on and an active experience with concepts and techniques.

In the first assignment, the student implements different validation metrics and test it on a pre-processed dataset of fingerprints.

Subsequently, based on a personal preference, two assignments are selected from the following three options:

1. Implementing a non-standard key point based fingerprint matching algorithm, an iris recognition system and a combination of both. 

2. Implementing and testing a face detection and face recognition algorithm.

3. Assessing and explaining a paper from the recent (within the last 5 years) international scientific literature on topics related to biometric systems.

Course material

Python Scripting Language, jupyter notebooks, google colab and online references

Explanation

The evaluation is based on the (individual) assignments. 

For the programming assignments Python notebooks and summary reports needs to be handed in.

For the literature investigation a paper assessment report needs to be handed in.

An online individual session is planned to discuss the assignments submitted and the content of the lectures.

Content

The motivation for the course lies in the urgent need for computer programs that assist people in digesting masses of unstructured information composed of text and other media. We need information retrieval technology when, for instance, we find information on the World Wide Web, in repositories of news and blogs, in biomedical document bases, or in governmental and company archives. Moreover, emails, tweets, other messages and advertisements are searched and filtered. Various techniques of content recognition, recommendation and linking play an increasing role and allow generating content models of the documents or messages, that effectively match the personalized information needs of users. We witness a current interest in capturing dynamic changes in the data and in modeling dynamic interactions with users. The proliferation of wireless and mobile devices such as mobile phones has additionally created a demand for effective and robust techniques to index, retrieve and summarize information.

 
The lectures treat the following topics:
 

1. Introduction
 

 2. Advanced representations

 Law of Zipf

Matrix factorization, latent semantic analysis (LSA), training with singular value decomposition

Probabilistic latent semantic analysis (pLSA), latent Dirichlet allocation (LDA), training with Expectation Maximization (EM) algorithms, Markov chain Monte Carlo (MCMC) methods such as Gibbs sampling, and with variational inference

Embeddings obtained with neural networks

3. Retrieval and search models

Algebraic models: vector space models

Probabilistic models: language retrieval models and Bayesian networks

Neural network models

4. Learning to rank

Relevance feedback, personalized and contextualized information needs, user profiling

Pointwise, pairwise and listwise approaches

Structured output support vector machines, loss functions, most violated constraints

End-to-end neural network models

Optimization of retrieval effectiveness and of diversity of search results

5. Dynamic retrieval and recommendation

Static versus dynamic models

Markov decision processes

Multi-armed bandit models

Modelling sessions

Online advertising

6. Multimedia information retrieval

Multimedia data types and features

Concept detection

Cross-modal indexing of content: latent Dirichlet allocation and deep learning methods

Cross-modal and multimodal retrieval and recommendation models

Illustrations with spoken document, image, video and music search

7. Web search

Web search engines, crawler-indexer architecture, query processing

Link analysis retrieval models: PageRank, HITS, personalized PageRank and variants

Behavior and credibility based retrieval models

Social search, mining and searching user generated content

8. Scalability of Web search 

Data structures and search techniques

Inverted files, nextword indices, taxonomy indices, distributed indices

Compression

Learning of hashing functions, cross-modal hashing

Scalability and efficiency challenges

Architectural optimizations

9. Clustering

Distance and similarity functions in Euclidean and hyperbolic spaces, proximity functions

Sequential and hierarchical cluster algorithms, algorithms based on cost-function optimization, number of clusters

Term clustering for query expansion, document clustering, multiview clustering

10. Categorization

Feature selection, naive Bayes model, support vector machines, (approximate) k-nearest neighbor models

Deep learning methods

Multilabel and hierarchical categorization

Convolutional neural network (CNN) based hierarchical categorization

11. Summarization

Document segmentation, maximum marginal relevance

Summarization based on latent Dirichlet allocation models and long short-term memory (LSTM) networks

Abstractive summarization with attention models

Multidocument summarization, search results fusion and visualization 

12. Question answering and conversational agents in search and recommendation

Retrieval based question answering

Deep learning methods including attention models

Cross-modal question answering

E-commerce search and recommendation

13. Evaluation measures and methodology

Recall, precision, F-measure, mean average precision, discounted cumulative gain, mean reciprocal answer rank, accuracy, confusion matrix, ROC curve, normalized mutual information, mean absolute error, root mean square error, pyramid method, inter-annotator agreement, test collections

14. Discussion of interesting research projects

15. Invited lecture by representative of an important company

In 2006-2007: Thomas Hofmann, Director of Engineering, Google Zurich European Engineering Centre, Switzerland; in 2007-2008: Ronny Lempel, director of Yahoo! research, Israel; in 2008-2009: Stephen Robertson, senior researcher at Microsoft Research Cambridge, UK and one of the founders of probabilistic modeling in information retrieval; in 2009-2010: Gregory Grefenstette, Chief Science Officer, Exalead, France; in 2010-2011: Mounia Lalmas, visiting senior researcher at Yahoo! Labs Barcelona, Spain; in 2011-2012: Jakub Zavrel, CEO and founder of TextKernel, The Netherlands; in 2012-2013: Massimiliano Ciaramita, senior research scientist at Google, Zürich, Switzerland; in 2013-2014: Alex Graves, senior research scientist at Google DeepMind, London, UK; in 2014-2015: Fabrizio Silvestri, Senior Scientist at Yahoo Labs, Barcelona; in 2015-2016: Roi Blanco, Senior Scientist at Yahoo Labs, London; in 2016-2017: Holger Schwenk, research scientist at Facebook AI Research, France and Dani Yogatama, research scientist at Google DeepMind, London, UK; in 2017-2018: Enrique Alfonseca, research tech leader at Google AI, Zurich; in 2020-2021: Florian Strub, senior researcher at Google Deepmind, and in 2021-2022: Rylan Conway, applied scientist at Amazon Seattle.

Course material

Course material is available on the Toledo-platform of the K.U.Leuven. The following books offer background to the course material:
Baeza-Yates, R. & Ribeiro-Neto, B. (2011). Modern Information Retrieval: The Concepts and Technology behind Search (2nd edition). Harlow, UK: Pearson.
Büttcher, S., Clarke, C.L.A. & Cormack, G.V. (2010). Information Retrieval: Implementing and Evaluating Search Engines. Cambridge, MA: MIT Press. 
Manning, C.D., Raghaven, P. & Schütze, H. (2009). Introduction to Information Retrieval. Cambridge University Press.
Moens, M.-F. (2006). Information Extraction: Algorithms and Prospects in a Retrieval Context (International Series on Information Retrieval 21). Berlin: Springer.

 

Format: more information

Interactive lectures.

Is also included in other courses

H02C8B : Information Retrieval and Search Engines

Content

• Exercise session on latent semantic models, probabilistic and vector models
• Exercise session on learning to rank
• Exercise session on dynamic retrieval
• Exercise session on compression
• Exercise session on categorization and clustering
• Exercise session on link based and multimodal models

Course material

Exercises and answers are available via the Toledo platform. 

Format: more information

Interactive exercise sessions in small groups.

Is also included in other courses

H02C8B : Information Retrieval and Search Engines

Explanation

Theory exam (grading: 50 %): Written, open book.

Exercise exam (grading: 50 %): Written, open book.

Content

• articulatory phonetics
• introduction to phonology and the sounds of the world's languages
• phonetic transcription
• acoustic phonetics, segmental (formants and temporal cues), and suprasegemental (rhythm, prosody, emphasis, etc)
• effects of coarticulation and assimilation
• context effects (speaker and speaking rate, etc);
• trading cues
• human perception (of robust spectral and temporal cues, normalisation, etc)
• human perception of vowels and consonants (categorical perception)
• human perception of suprasegmental cues
• bimodality (McGurk)
• the problem of lack of invariance and difficulty of segmentation
• speech synthesis: different types (parametric, concatenative), high & low level synthesis, problems and solutions
• text-to-speech systems
• applications with speech synthesis: different needs and necessities

Course material

• slides & handouts (via Toledo)
• Praat software (www.praat.org)
• speech materials (Toledo)

Language of instruction: more information

Classes are taught in English because the majority of the students are from foreign.

Format: more information

Students are expected to participate actively during class. Discussions are highly encouraged.

Is also included in other courses

G0D23A : Speech Science

Content

A series of different exercises will be given, including

• phonetic transcription
• speech segmentation
• formant analyses based on FFT, LPC, spectrogram reading
• analyses of fundamental frequency
• pitch analysis (autocorrelation)
• intensity analyses
• manipulation of the speech signal (eg PSOLA)
• analysis of articulation and speech rate
• spectrogram reading
• filtering
• making speech sounds
• text grid (for linguistic analyses)
• speech synthesis
• ...

Course material

All exercises can be done 'at home' using Praat (www.praat.org, Univ. of Amsterdam, Boersma & Weenink), a versatile and free software program. Solutions will be discussed in class.
Exercises will be disseminated via Toledo.

Language of instruction: more information

All classes will be given in English due the large number of foreign students attending the program.

Format: more information

Excercises are done at home using the Praat software to understand the nature of the time-varying speech signal, especially in relation to automatic speech recognition and speech synthesis.

Is also included in other courses

G0D23A : Speech Science

Explanation

After the course classes, but prior to the exam the student will be presented with a take-home exercise based on the series of exercises discussed before. This take-home exercise needs to be handed in before taking the exam and counts for 4/20 points.

The written exam during the examination period counts for 16/20 points and consists of both kowledge and reasoning questions. Students who have difficulty writing in English can explain orally (after a written preparation).

Content

High-level contents and materials

• Lecture 1: Introduction
• Lecture 2: Problems, representation, and variation
• Lecture 3: Population management
• Online module 1: Local search operators 
• Online module 2: Multiobjective optimization and diversity promotion
• Online case study modules: Hands-on programming exercises

Detailed contents

Basics of evolutionary algorithms

• Exploration versus exploitation
• Computational and optimization problems
• Objective function
• Representation
• Constraints
• Variation operators
• Selection and elimination operators
• Hyperparameter self-adaptivity

Local search operators

• Steepest descent
• Monte Carlo sampling
• k-opt

Multi-objective optimization and diversity promotion

• Crowding
• Island model
• Fitness sharing
• Scalarization (fixed tradeoff)
• Pareto front

Course material

Study cost: 51-75 euros (The information about the study costs as stated here gives an indication and only represents the costs for purchasing new materials. There might be some electronic or second-hand copies available as well. You can use LIMO to check whether the textbook is available in the library. Any potential printing costs and optional course material are not included in this price.)

Slides, online videos, textbook (e-book).

Content

The exercises consists of four guided sessions during which (most of) the group phase of the project is executed. For the group phase you will be assigned to balanced groups of 3-5 students. Attendance to the exercise sessions is mandatory, and non-participation results in a final mark of NA.

In the four exercise sessions, a basic evolutionary algorithm will be designed. The content of the sessions are:

• Design of an elementary evolutionary algorithm
• Computer implementation of an evolutionary algorithm in Python
• Experimentation with the algorithm and reporting
• Reading reports and providing peer feedback to other groups

Content

The students undertake a two-phase project. The first phase is a group work in which the students analyze a model problem and design, implement, and test an evolutionary algorithm in the Python programming language. This phase is concluded by a peer feedback assignment in which the students analyse one or more designs from other teams and provide feedback on them. The second phase is performed individually by each student, in which they analyze the results from their group phase, and based on the acquired insights, design, implement, and test improved variation and local search operators, selection mechanisms, diversity promotion schemes, among others. The students report the results of their analysis (results, computational performance, strengths and weaknesses, among others) via two reports, one for each phase. 

The reports will be discussed on the oral exam and, along with additional general questions, constitute the majority of the grade for this course. See the evaluation section for more details.

Explanation

The evaluation is oral without written preparation. The exam consists of a short discussion of the project and open theoretical questions about the course with the project as potential case study.

The score of the exam is the sum of the score of the group phase of the project, the individual phase of the project, and the exam.

If the student fails to participate in one of the components of the project (exercises, peer feedback, individual programming, individual report), the outcome of the exam is NA.

Information about retaking exams

The retake exam consists of an updated project assignment for the take-home exam and an oral exam without written preparation. The setup of the exam and scoring is the same as in the first examination period, except that the group phase must be completed individually if its previous outcome was NA. The updated assignment will be uploaded after the closing of the second examination period to Toledo. The deadline of the project will be at least one week before the opening of the third examination period.

The score of the exam is determined according to the same modalities as in the first exam attempt.

Content

Bayesian probability theory: modelling, inference, reasoning, decision making

Graphical models -- Bayesian networks, Markov Networks and Factor Graphs

Independence in graphical models

Inference algorithms 

Hidden and observable parameters

Learning

Dynamic systems (such as Hidden Markov Models and Kalman Filters)

Combining logic with graphical models

Applications

Course material

The  course is based on (selected) parts of David Barber's forthcoming book on Bayesian Reasoning and Machine Learning, available from http://www.cs.ucl.ac.uk/staff/d.barber/brml and some additional materials.

Content

There are around 6 exercise sessions (mostly with pen and paper) on various aspects of uncertainty reasoning and graphical models.

Content

Each year students have to make one or more assignments and hand in their solution. This can take the form of traditional exercises or of a small project with software for graphical models.

Format: more information

The project consists of one or more assignments, possibly involving tasks with implementations of graphical models.

Explanation

The evaluation consists of 
    closed book exam (with the use of a formularium, during the exam period, by far the most important part of the evalution), and 
    reports on the assignments.

Information about retaking exams

The exam can be retaken but the assignments cannot be retaken.