Your first LLM app

A first LLM app is only a few lines — create a client, send messages, print the reply. A useful one adds a little structure around those lines: configuration that stays out of your code, one function that every feature calls, usage and cost tracking, and retries for the errors every API eventually returns. This lesson builds that foundation, which the rest of the course reuses.

Last updated: 01 Nov, 2026

The minimal app

Definition · Chat Completions request

An HTTP request containing a model name and a list of messages (each with a role — system, user, or assistant — and content). The response contains the generated message plus metadata such as token usage.

Python

import os

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()                      # reads OPENAI_API_KEY from a .env file
client = OpenAI()                  # picks up OPENAI_API_KEY automatically

response = client.chat.completions.create(
    model=os.getenv("MODEL_NAME", "gpt-4.1-mini"),
    messages=[
        {"role": "system", "content": "You are a concise assistant for developers."},
        {"role": "user", "content": "Explain tool calling in one sentence."},
    ],
    temperature=0,
)
print(response.choices[0].message.content)
print(response.usage)

Tool calling lets a model ask your code to run a named function with structured arguments and then use the result in its answer. CompletionUsage(completion_tokens=27, prompt_tokens=31, total_tokens=58, ...)

Model replies vary, so outputs from live calls in this course are examples. Three things to notice: the API key never appears in code, the model name comes from configuration, and the response carries usage — the tokens you pay for.

Configuration and secrets

Definition · Environment variable

A named value provided by the operating system or a .env file when the program starts. Use them for secrets (API keys) and for settings that differ between environments (model names, timeouts).

.env

# .env  (add this file to .gitignore - never commit it)
OPENAI_API_KEY=sk-...
MODEL_NAME=gpt-4.1-mini
REQUEST_TIMEOUT=30

Watch out

Never paste API keys into source files, notebooks you share, or front-end JavaScript. Anyone with the key can spend your money. If a key leaks, revoke it in the provider dashboard immediately.

One function for every call

Call the model from exactly one place. Then logging, retries, cost tracking, and switching providers happen in one function instead of fifty. Inject the function that actually talks to the model, so you can test with a fake one.

Python

import time
from dataclasses import dataclass

PRICE_PER_MILLION = {"input": 0.40, "output": 1.60}   # example prices in USD - check your provider


@dataclass
class Reply:
    text: str
    input_tokens: int
    output_tokens: int
    seconds: float

    @property
    def cost(self) -> float:
        return (self.input_tokens * PRICE_PER_MILLION["input"]
                + self.output_tokens * PRICE_PER_MILLION["output"]) / 1_000_000


def fake_complete(messages: list[dict]) -> tuple[str, int, int]:
    """Stands in for the API: returns (text, input_tokens, output_tokens)."""
    return "Tool calling lets a model ask your code to run a function.", 31, 14


def ask(prompt: str, system: str = "You are a concise assistant.", complete=fake_complete) -> Reply:
    messages = [{"role": "system", "content": system}, {"role": "user", "content": prompt}]
    start = time.perf_counter()
    text, tokens_in, tokens_out = complete(messages)
    return Reply(text, tokens_in, tokens_out, time.perf_counter() - start)


reply = ask("Explain tool calling in one sentence.")
print(reply.text)
print(f"tokens: {reply.input_tokens} in / {reply.output_tokens} out | cost: ${reply.cost:.6f}")

Tool calling lets a model ask your code to run a function. tokens: 31 in / 14 out | cost: $0.000035

The live version of complete wraps the SDK call and reads usage from the response:

Python

def openai_complete(messages: list[dict]) -> tuple[str, int, int]:
    response = client.chat.completions.create(
        model=os.getenv("MODEL_NAME", "gpt-4.1-mini"), messages=messages, temperature=0,
        timeout=float(os.getenv("REQUEST_TIMEOUT", "30")),
    )
    usage = response.usage
    return response.choices[0].message.content, usage.prompt_tokens, usage.completion_tokens


reply = ask("Explain tool calling in one sentence.", complete=openai_complete)

Errors you must handle

Error	HTTP status	Meaning	What to do
Authentication	401	Missing or invalid API key	Fail fast with a clear message; do not retry
Bad request	400	Invalid parameters, or the input is too long for the context window	Fix the request; do not retry unchanged
Rate limit	429	Too many requests or tokens per minute	Retry with exponential backoff
Server error	500–503	Provider-side problem	Retry with backoff
Timeout	—	No response within your time limit	Retry a limited number of times

Definition · Exponential backoff

A retry strategy that waits longer after each failure — for example 1 s, 2 s, 4 s — usually with a little random jitter so many clients do not retry at the same instant. It gives an overloaded service time to recover.

Python

import random
import time


class RateLimitError(Exception):
    pass


attempts_seen = {"n": 0}


def flaky_complete(messages: list[dict]) -> tuple[str, int, int]:
    """Fails twice with a rate limit, then succeeds - like a busy API."""
    attempts_seen["n"] += 1
    if attempts_seen["n"] < 3:
        raise RateLimitError("429 Too Many Requests")
    return "Recovered after retries.", 12, 4


def with_retries(fn, *args, max_attempts: int = 4, base_delay: float = 0.01):
    for attempt in range(1, max_attempts + 1):
        try:
            return fn(*args)
        except RateLimitError as error:
            if attempt == max_attempts:
                raise
            delay = base_delay * 2 ** (attempt - 1)
            print(f"attempt {attempt} failed ({error}); waiting {delay * 1000:.0f} ms")
            time.sleep(delay * (1 + random.random() * 0.1))   # add jitter


text, _, _ = with_retries(flaky_complete, [{"role": "user", "content": "hi"}])
print(text)

attempt 1 failed (429 Too Many Requests); waiting 10 ms attempt 2 failed (429 Too Many Requests); waiting 20 ms Recovered after retries.

Note

The OpenAI SDK already retries rate limits and server errors (twice by default; set max_retries on the client). Write your own wrapper only when you need custom behaviour, such as falling back to another model, and never retry authentication or bad-request errors.

A small command-line app

app.py

"""app.py - ask a question from the terminal."""
import sys

from dotenv import load_dotenv
from openai import OpenAI, APIError, AuthenticationError

load_dotenv()
client = OpenAI(max_retries=3, timeout=30)


def main() -> int:
    question = " ".join(sys.argv[1:]) or "What is an AI agent?"
    try:
        response = client.chat.completions.create(
            model="gpt-4.1-mini",
            messages=[{"role": "user", "content": question}],
            temperature=0,
        )
    except AuthenticationError:
        print("Check OPENAI_API_KEY in your .env file.", file=sys.stderr)
        return 1
    except APIError as error:
        print(f"The model API failed: {error}", file=sys.stderr)
        return 2
    print(response.choices[0].message.content)
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Terminal

python app.py "What is an AI agent in one sentence?"

An AI agent is a system that uses a language model to decide which actions to take, step by step, to achieve a goal.

Try it

Add a --model option to app.py and print the cost of each call using PRICE_PER_MILLION.
Make with_retries fall back to a second model name after the final failed attempt.
Log every call (time, model, tokens, cost) as one JSON line in calls.jsonl.

Key takeaways

Keep keys and model names in environment variables, never in code.
Route every model call through one function so logging, retries, and cost tracking live in one place.
Track input and output tokens on every call — they are your bill.
Retry rate limits, timeouts, and server errors with exponential backoff; fail fast on authentication and bad requests.
Inject the model-calling function so the rest of the app can be tested with a fake.