What is Generative AI

The goal of this section is to explain and demystify all these acronyms that we are now surrounded by: AI, GPT, ML, LLM. While it might be tempting to jump straight into using chatbots like ChatGPT, Claude, Gemini, or others, having a conceptual understanding of the underlying technologies will help you become a more effective user and enable you to critically evaluate claims about their capabilities and limitations.

AI: Artificial Intelligence

There is no strict, uniformly accepted definition of “artificial intelligence”, but generally, it refers to technologies capable of performing tasks that would otherwise require human intelligence. Interestingly, what is considered smart enough to qualify as “artificial intelligence” constantly changes as previously impressive technological achievements become commonplace.

All recent advances in artificial intelligence have been driven by machine learning (ML), which is discussed in the next section. For that reason, AI and ML became essentially synonymous. Curiously, after the popularity of chatbots like ChatGPT, many existing machine learning tools were rebranded as AI tools without changing anything substantial. This is technically correct as machine learning is a subfield of AI, but this trend suggests that we should be careful about the actual meaning when encountering AI-branded products and services.

Check the video to learn more about the term AI and its connection to machine learning. Feel free to watch it at 1.25x speed if you prefer.

ML: Machine Learning

Machine learning refers to computer algorithms that work differently from traditional programming: instead of specifying exactly what has to be done through rules and instructions, their output depends on data called training data. This approach allows algorithms to automatically detect patterns in the training data and apply them to new, unseen data sets.

Most commonly, machine learning is used for classification and ranking tasks, such as recommending products you might be interested in, detecting potential spam messages, or recognising a person from a photo.

Check the video to get a conceptual understanding of how machine learning works. Feel free to watch it at 1.25x speed if you prefer.

At a certain point, researchers realised that the same techniques might be used to generate new content, hence the term “Generative AI”. A significant breakthrough in this direction came with the development of Generative Adversarial Networks (GANs). In 2018, NVIDIA researchers demonstrated that GANs could create remarkably realistic images of human faces that never existed.A Style-Based Generator Architecture for Generative Adversarial Networks (Karras et al., 2018) 

Similar to generating high-quality images, it soon became possible to generate high-quality text thanks to large language models, which are discussed in the next section.

LLMs: Large Language Models

Large language models are essentially machine learning algorithms that solve a very specific task: predicting the next word in a sentence, which allows them to generate larger texts word by word. Thanks to vast amounts of training data—modern large language models are essentially trained on the whole internet—and substantial computational resources, we are now able to train models that produce remarkably consistent texts.

Some researchers realised that very good language models can serve as a sort of general artificial intelligence tool. Imagine, for example, a model tasked with continuing the sentence ‘The capital of France is’. A good LLM would continue with ‘Paris’, not with ‘perish’ or ‘Berlin’, and thus can effectively answer geography questions. Curiously, before first GPT model was introduced in 2018, many researchers didn‘t believe that LLMs would be good enough to serve as general AI. Even experts in machine learning and natural language processing were caught by surprise by the performance of ChatGPT.

Check the video to get a conceptual understanding of how large language models work. As an experiment, this lecture is co-presented with my AI assistant:

Note on terminology

While widely used, the term “AI” is often too general to be useful, and in many cases “machine learning” or “large language models” should be preferred. However, as the term “Generative AI” has become so widespread in both academic literature and public discussions, we’ll continue to use it throughout this course.

You might also encounter the term AGI, which stands for “Artificial General Intelligence”. The main idea behind the term is that unlike traditional AI, AGI is not limited to a specific task. AGI is often described as a future state of AI, a point in time when AI would have a profound societal impact—utopian or catastrophic depending on one‘s perspective.

Despite its widespread use and inclusion in OpenAI’s vision statement, the term doesn’t seem to be very useful practically. On one hand, one might argue that modern chatbots are already performing such a wide range of tasks that we have reached AGI. On the other hand, there is no need to achieve some specific point in the development of AI for it to have a profound societal impact. “Almost AGI” or even an AI that is exceptionally good at one specific task could be extremely disruptive. If you are interested in this topic, you can read about attempts to operationalise the definition of AGILevels of AGI for Operationalizing Progress on the Path to AGI (Morris et al., 2023) .