LLM Fundamentals

LLM Fundamentals is the mental model behind every chatbot, copilot, RAG system, and agent. A large language model turns your text into tokens, predicts the next token over and over, and is exposed to you as an HTTP API that takes a list of messages. Once you understand those pieces — and the cost, latency, and limits attached to them — you can build with any model from any provider and debug it when it misbehaves.

Last updated: 01 Nov, 2026

What this course is about

Definition · Large language model (LLM)

A neural network, trained on a very large amount of text, that predicts the next piece of text (a token) given the text so far. By repeating that prediction it can answer questions, write code, summarise documents, translate, and follow instructions.

Most people use LLMs through a chat window and treat them as magic. AI engineers cannot afford to. When a model invents a fact, forgets something said ten messages ago, returns broken JSON, costs ten times what you expected, or times out under load, you need to know why — and the answer always comes back to a handful of fundamentals. This course teaches exactly those fundamentals, with definitions, diagrams, runnable Python, and real API calls.

The one mental model to remember

Every lesson in this course zooms in on one part of this loop:

Mental model

your messages (system + user + history)
        │
        ▼
  ┌──────────────┐
  │  tokenizer   │  text  ->  token ids        (lesson: Tokens and context windows)
  └──────────────┘
        │
        ▼
  ┌──────────────┐
  │  the model   │  token ids  ->  a score for every possible next token
  └──────────────┘                              (lesson: Next-token prediction)
        │
        ▼
  ┌──────────────┐
  │   sampler    │  scores  ->  pick ONE token (temperature, top-p)
  └──────────────┘                              (lesson: Temperature and sampling)
        │
        ├── append the token and repeat until a stop condition
        ▼
  assistant reply  +  usage (tokens, cost, latency)
                                                (lessons: API call, Messages, Cost)

What you will learn

Lesson	What you learn	Why it matters
What is an LLM	AI vs ML vs deep learning, transformers, pretraining and fine-tuning, base vs chat models, strengths and limits	Sets realistic expectations and explains hallucinations.
Tokens and context windows	Tokenizers, BPE, counting tokens with `tiktoken`, context budgets, truncation	Tokens decide cost, speed, and what the model can “see”.
Next-token prediction	Logits, softmax, the generation loop, stop conditions, training loss, prefill vs decode	Explains why prompts work and why models ramble or cut off.
Your first API call	SDK setup, keys, the request/response shape, providers, errors, streaming	Turns understanding into working code.
Messages and roles	system, developer, user, assistant, and tool roles; stateless chat; history management	Every chat app, RAG system, and agent is built on messages.
Temperature and sampling	Temperature maths, top-k, top-p, penalties, seeds, settings per task	Controls creativity vs reliability.
Cost, latency, and limits	Pricing, TTFT, throughput, rate limits, logging, caching, batching	What separates a demo from a production system.

What you will be able to do

Explain to a colleague, precisely, what happens between pressing Enter and seeing a reply.
Count tokens before you send a request and keep conversations inside the context window.
Call OpenAI, Anthropic, Gemini, Groq, or a local Ollama model from Python with proper error handling.
Build a multi-turn chat loop that manages its own history.
Choose temperature and sampling settings deliberately for extraction, chat, and creative tasks.
Log tokens, cost, and latency for every call, and stay within rate limits.

Prerequisites

Basic Python: variables, functions, lists, dicts, and installing packages in a virtual environment. The Python for AI Developers course covers all of it.
An API key (optional but recommended) from any chat provider — OpenAI, Anthropic, Google Gemini, Groq, or OpenRouter. If you prefer not to pay, you can run a free local model with Ollama; the API lesson shows how.
No maths background needed. The few formulas (softmax, temperature) are explained in plain English and demonstrated with code.

Setup for this course

Terminal

python3 -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
python -m pip install openai tiktoken python-dotenv numpy

Most code examples run fully offline — toy models, tokenizers, and simulations — so you can learn the concepts without spending anything. Examples that call a real model are clearly marked, and their output will vary on every run.

Key terms you will meet

Term	Meaning
Token	The unit of text a model reads and writes — often a word, part of a word, or punctuation.
Tokenizer	The component that converts text to token ids and back.
Context window	The maximum number of tokens (input + output) a model can handle in one request.
Parameters	The learned numbers (weights) inside a neural network; modern LLMs have billions.
Logits	Raw scores the model gives every possible next token before they become probabilities.
Softmax	The function that turns logits into probabilities that sum to 1.
Temperature	A setting that sharpens or flattens the probability distribution before sampling.
Inference	Running a trained model to generate output (as opposed to training it).
Hallucination	Fluent, confident output that is factually wrong or unsupported.
Rate limit	The maximum requests or tokens per minute a provider allows your account.

Note

Model names, prices, and context sizes change every few months. This course teaches the concepts that stay stable and shows you where to look up current numbers. Wherever a specific figure appears, treat it as an example and check your provider's documentation.

Course syllabus

Work through the lessons in order — each one builds on the last. Your progress is saved in this browser as you mark lessons done.

Syllabus