Understanding Transformers

This is a 2-day course on the topic of transformers architecture, where we will dive into the theory and application use cases of this powerhouse in modern AI. At the beginning of the course we will build up an intuition for neural networks and generative AI in general, before we dive into the deeper details of transformers, such that we gain a gradual and intuitive understanding of the AI landscape overall. After going through the theoretical aspects, the practical task will be to build a mini-chatgpt system from scratch, in a bottom-up manner, illustrating all the building blocks in code.

Course structure:

First day: 3 hour session from 10:00AM to 1:00PM

There will be a half-hour break after the first session hour, which you may use to install the software libraries for the practical session on Day 2.

Second day: 3 hour session from 10:00AM to 1:00PM

There will be a half-hour break after the first session hour.

Learning goals

Day 1: Theoretical session

• Introduction to neural nets in general
• Introduction to generative AI / genAI
• Theory and applications of transformers

Day 2: Practical session

• Build all the components of a transformer that we touched on in Day 1
• Assemble a transformer out of all the building block components

Setup a mini-chatgpt system that emulates speech via predicting next characters.

Prerequisites

• It is recommended to have at least a beginner level of Python (see the course “First steps in Python”)
• It is recommended to have a basic understanding of linear algebra and statistics (see the course “Introduction to Statistics”)

Target group

The course is targeted at a heterogeneous audience with various levels of AI knowledge, including researchers that may or may not have had exposure to AI topics beforehand, but still want to learn about the inner workings of transformers to apply them in their research tasks.

This course is free of charge.