1 Introduction: LLMs revolutionize (almost) everything! #
1.1 AI is Changing the World #
The availability of vast amounts of data and computing power has enabled applications of Artificial Intelligence (AI), especially those based on deep learning, to break through in many areas:
- Image processing: character recognition, face detection, identification of animals / plants / places …
- Autonomous driving
- Recommender: movies, music, social media posts, …
- Spam filters
- Autocomplete (e.g. on mobile phones)
- Automatic text translation
- Chatbots (ChatGPT): question answering, text summaries, …
- Generation of images and videos, coding, audio, …
In this lecture, we will focus on large language models (LLMs) and, in particular on text generation using LLMs.
However, we will also cover techniques for natural language processing (NLP) preceding LLMs, e.g. for autocompletion (e.g. when typing on smart phones).
Figure: Autocompletion (here in WhatsApp on iOS)
1.2 A Forerunner of ChatGPT: Google’s Transformer Paper #
With the publication of “Attention is all you need”, researchers at Google Brain and Google Research set a new direction for NLP in 2017.
Figure: Abstract of 2017 paper Attention is all you need (https://arxiv.org/abs/1706.03762)
The paper specifically addressed English–German and English–French machine translation. However, the new concepts are now used by all leading language models. Specifically, recurrent and convolutional networks (RNNs and CNNs) were abandoned and replaced by the transformer architecture, using self-attention mechanisms (initially in an “encoder” and a “decoder” structure).
The field remains highly active: as of November 2025, Google Scholar counts more than 200,000 publications that cite the transformer paper.
Figure: Citation counts for Attention is all you need (https://arxiv.org/abs/1706.03762)
1.3 Investors are betting on LLMs (and AI in general) #
Since the public release of ChatGPT in November 2022, leading AI companies such as OpenAI and Antropic have reached enormous valuations (compared to their income). This is also true for NVIDA, the leading supplier of AI accelerators (i.e. data center GPUs).
| Company | Valuation 11/2025 | Comments |
|---|---|---|
| OpenAI | 500 B | (2023: 90 B; 2020: 10 B) |
| Anthropic | > 200 B | (3/2025: 60 B) |
| Mistral | > 10 B | (6/2024: 6 B) |
| NVIDIA | > 4.000 B | (10/2022: 300 B) |
Figure: Percent change in stock value for Magnificient Seven: since the release of ChatGPT, NVIDIA went far ahead.
Source: https://www.reuters.com/business/nvidia-poised-record-5-trillion-market-valuation-2025-10-29/
1.4 Huge innovations are happening every day - stay tuned! #
As evidenced by the number of citations to the “transformer paper”, the field is extremely active with follow-up papers being published essentially by the hour.
Non-LLM-specialists need secondary sources for staying up to date. One possibility is LinkedIn (esp. recommended: posts/comments by R. Peinl , Nir Diamant, and Andrew Ng).
Last updated: 2025-12-10 13:15