Your first API call
Calling an LLM is an HTTP request with a JSON body, wrapped by an official SDK. In this lesson you will make your first real call, understand every field of the request and the response, write a reusable helper with proper error handling, stream tokens as they are generated, and point the same code at different providers — including a free model running on your own laptop.
Definitions
A web service that runs a language model for you. You send an HTTPS request containing the model name and your messages; it returns the generated text plus metadata such as token usage.
Software Development Kit — an official library (for example openai or anthropic on PyPI) that wraps the raw HTTP API in Python classes and adds authentication, retries, timeouts, streaming, and typed responses.
The widely copied request format introduced by OpenAI: POST /v1/chat/completions with a model and a list of messages. Many providers offer an “OpenAI-compatible” version of this endpoint, so one SDK can talk to all of them.
Step 1 — Get an API key and keep it secret
- Create an account with a provider (OpenAI, Anthropic, Google AI Studio, Groq, OpenRouter…).
- Add billing or use the free tier, and set a monthly spending limit.
- Create an API key and copy it once — most dashboards will not show it again.
- Store it in a git-ignored
.envfile, never in code.
OPENAI_API_KEY=sk-...
MODEL_NAME=gpt-4.1-mini
Step 2 — Install the SDK
python -m pip install openai python-dotenv
Step 3 — Make the call
import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI() # reads OPENAI_API_KEY from the environment automatically
response = client.chat.completions.create(
model=os.getenv("MODEL_NAME", "gpt-4.1-mini"),
messages=[
{"role": "system", "content": "You are a concise AI tutor."},
{"role": "user", "content": "Explain what an embedding is in one sentence."},
],
temperature=0.2,
max_tokens=100,
)
print(response.choices[0].message.content)
print("finish:", response.choices[0].finish_reason)
print("tokens:", response.usage.prompt_tokens, "in /", response.usage.completion_tokens, "out")
python first_call.py
Your wording and token counts will differ — the model generates fresh text each time. What stays the same is the shape of the code and the response.
Understanding the request
| Parameter | Required | Meaning |
|---|---|---|
model | Yes | Which model to run. Names are provider-specific and change over time — check the provider's model list. |
messages | Yes | The conversation: a list of {"role", "content"} dicts (next lesson). |
temperature | No | Randomness, typically 0–2. Low for factual tasks, higher for creative ones. |
max_tokens / max_completion_tokens | No (required on Anthropic) | Upper limit on generated tokens. Protects you from runaway cost. |
stop | No | Up to a few strings that end generation when produced. |
response_format | No | Ask for JSON or a JSON schema (structured outputs). |
stream | No | Send tokens as they are generated. |
tools | No | Functions the model may call (Agents course). |
Understanding the response
ChatCompletion(
id="chatcmpl-...",
model="gpt-4.1-mini-2025-04-14", # exact version that served you
choices=[
Choice(
index=0,
message=ChatCompletionMessage(role="assistant", content="An embedding is ..."),
finish_reason="stop",
)
],
usage=CompletionUsage(prompt_tokens=31, completion_tokens=27, total_tokens=58),
)
| Field | Use it for |
|---|---|
choices[0].message.content | The generated text — what you show the user. |
choices[0].finish_reason | Why it stopped: stop, length, tool_calls, content_filter. |
usage | Exact token counts for cost tracking. |
model | The precise model version, for logging and reproducibility. |
id | Request identifier for support tickets and tracing. |
Do not print(response) and try to read the answer out of the long object dump. Always extract response.choices[0].message.content, and check finish_reason — if it is length, the answer was cut off.
OpenAI also offers a newer Responses API (client.responses.create(model=..., input=...), with the text in response.output_text) that adds built-in tools and server-side conversation state. Chat Completions remains the most portable format across providers, so this course uses it; the concepts transfer directly.
A reusable helper
Wrap the call once, with sensible defaults, so the rest of your code just asks questions:
import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI(timeout=60, max_retries=3)
DEFAULT_MODEL = os.getenv("MODEL_NAME", "gpt-4.1-mini")
def ask(prompt: str, system: str = "You are a helpful assistant.",
model: str = DEFAULT_MODEL, temperature: float = 0.2,
max_tokens: int = 500) -> str:
response = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": system},
{"role": "user", "content": prompt},
],
temperature=temperature,
max_tokens=max_tokens,
)
choice = response.choices[0]
if choice.finish_reason == "length":
print("warning: answer was truncated by max_tokens")
return choice.message.content or ""
if __name__ == "__main__":
print(ask("Give two bullet points on why tokens matter for cost."))
Handling errors
The SDK raises specific exception classes, so you can respond to each failure correctly. It already retries connection errors, 429, and 5xx responses with backoff (controlled by max_retries); you handle what remains.
import openai
from openai import OpenAI
client = OpenAI(max_retries=2, timeout=30)
try:
response = client.chat.completions.create(
model="gpt-4.1-mini",
messages=[{"role": "user", "content": "Hello"}],
)
print(response.choices[0].message.content)
except openai.AuthenticationError:
print("Invalid or missing API key - check OPENAI_API_KEY.")
except openai.RateLimitError:
print("Rate limit or quota exceeded - slow down or check billing.")
except openai.BadRequestError as err:
print("Request rejected (bad model name, too many tokens, ...):", err.message)
except openai.APIConnectionError:
print("Network problem - could not reach the API.")
except openai.APIStatusError as err:
print("API error", err.status_code)
| Exception | HTTP status | Typical cause |
|---|---|---|
AuthenticationError | 401 | Wrong, revoked, or missing key. |
PermissionDeniedError | 403 | No access to that model or region. |
NotFoundError | 404 | Misspelled or retired model name. |
BadRequestError | 400 | Invalid parameters or context length exceeded. |
RateLimitError | 429 | Too many requests/tokens per minute, or no credit left. |
InternalServerError | 5xx | Provider outage — retried automatically. |
APIConnectionError / APITimeoutError | — | Network failure or timeout. |
Streaming tokens as they arrive
Because models generate one token at a time, they can send each piece immediately. Streaming makes chat interfaces feel fast: users see the first words in a fraction of a second instead of waiting for the full answer.
from openai import OpenAI
client = OpenAI()
stream = client.chat.completions.create(
model="gpt-4.1-mini",
messages=[{"role": "user", "content": "List three uses of LLMs, one line each."}],
stream=True,
)
for chunk in stream:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
print()
Switching providers
Many providers implement the OpenAI-compatible Chat Completions endpoint. Change only the base_url, the key, and the model name:
import os
from openai import OpenAI
PROVIDERS = {
"openai": {"base_url": None, "key": "OPENAI_API_KEY", "model": "gpt-4.1-mini"},
"groq": {"base_url": "https://api.groq.com/openai/v1", "key": "GROQ_API_KEY", "model": "llama-3.3-70b-versatile"},
"openrouter": {"base_url": "https://openrouter.ai/api/v1", "key": "OPENROUTER_API_KEY", "model": "meta-llama/llama-3.3-70b-instruct"},
"gemini": {"base_url": "https://generativelanguage.googleapis.com/v1beta/openai/", "key": "GEMINI_API_KEY", "model": "gemini-2.5-flash"},
"ollama": {"base_url": "http://localhost:11434/v1", "key": None, "model": "llama3.2"},
}
def make_client(name: str) -> tuple[OpenAI, str]:
cfg = PROVIDERS[name]
api_key = os.getenv(cfg["key"]) if cfg["key"] else "ollama" # local needs any string
return OpenAI(base_url=cfg["base_url"], api_key=api_key), cfg["model"]
client, model = make_client("groq")
reply = client.chat.completions.create(
model=model, messages=[{"role": "user", "content": "Say hi in five words."}]
)
print(reply.choices[0].message.content)
Model names in this table are examples that change frequently — always copy current names from each provider's model list. Compatibility layers also vary: advanced features such as structured outputs or tool calling may behave slightly differently between providers.
Anthropic's native SDK
Anthropic's Messages API has a slightly different shape: the system prompt is a top-level parameter, max_tokens is required, and the text lives in a list of content blocks.
import anthropic
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY
message = client.messages.create(
model="claude-sonnet-4-5",
max_tokens=300,
system="You are a concise AI tutor.",
messages=[{"role": "user", "content": "What is a context window?"}],
)
print(message.content[0].text)
print(message.stop_reason, message.usage.input_tokens, message.usage.output_tokens)
Free option: run a model locally with Ollama
A free tool that downloads open-weight models and runs them on your own computer, exposing an OpenAI-compatible API at http://localhost:11434/v1. Nothing leaves your machine and there is no per-token cost.
# install from https://ollama.com, then:
ollama pull llama3.2 # ~2 GB download
ollama run llama3.2 "Hello" # quick test in the terminal
Then use make_client("ollama") from the providers example. Small local models are less capable than frontier APIs, but they are perfect for learning, prototyping, and private data.
Troubleshooting
| Symptom | Fix |
|---|---|
OpenAIError: The api_key client option must be set | The environment variable is missing — call load_dotenv() before creating the client, or export the key. |
404 model not found | Check the exact model name and that your account has access to it. |
429 insufficient_quota | Add credit or check billing — this is not a speed problem. |
context_length_exceeded | Trim history or documents, or lower max_tokens. |
| Answer ends mid-sentence | finish_reason == "length" — raise max_tokens. |
| Works locally, fails on the server | The server lacks the environment variable — configure secrets in your hosting platform. |
- Run
first_call.py, then change the system prompt to “Answer like a pirate” and compare. - Set
max_tokens=10and observefinish_reason. - Add timing with
time.perf_counter()and print latency next to token usage. - Install Ollama and run the same prompt against a local model.
- An LLM call is a POST with a model name and a list of messages; the SDK handles HTTP, auth, and retries.
- Read the answer from
choices[0].message.contentand always checkfinish_reasonandusage. - Keep keys in environment variables and set spending limits before experimenting.
- Catch the SDK's specific exceptions to handle auth, rate-limit, and bad-request errors differently.
- Stream for responsive UIs, and switch providers with
base_urlthanks to OpenAI-compatible endpoints.