Lists, dicts, and JSON
Almost all data in AI engineering takes the same shape: a list of records, where each record is a dictionary of named fields. Chat messages, retrieved documents, evaluation results, and API payloads all look like this — and they travel between programs as JSON. This lesson teaches Python's collection types in depth and how to move them in and out of JSON safely.
Python's four built-in collections
| Type | Syntax | Ordered | Mutable | Duplicates | Typical AI use |
|---|---|---|---|---|---|
list | [1, 2, 3] | Yes | Yes | Allowed | Documents, chunks, messages, scores |
tuple | (1, 2) | Yes | No | Allowed | Fixed pairs like (doc_id, score) |
set | {"a", "b"} | No | Yes | Not allowed | Unique tags, deduplicating ids |
dict | {"k": "v"} | Yes (insertion order) | Yes | Keys unique | Records, configs, API payloads |
Mutable means the object can be changed in place after it is created. Ordered means items keep a stable position you can rely on.
Lists
An ordered, mutable sequence of items, written with square brackets. Items can be of any type and are accessed by their position, called an index, which starts at 0.
Indexing and slicing
chunks = ["intro", "setup", "retrieval", "generation", "evals"]
print(chunks[0]) # first item
print(chunks[-1]) # last item
print(chunks[1:3]) # items 1 and 2 (end index is excluded)
print(chunks[:2]) # first two
print(chunks[-2:]) # last two
print(len(chunks)) # number of itemsA slice [start:end] includes start but excludes end. Slicing never raises an error for out-of-range values, while indexing a missing position raises IndexError.
Changing a list
messages = ["system prompt"]
messages.append("user: hi") # add one item to the end
messages.extend(["assistant: hello", "user: thanks"]) # add many
messages.insert(1, "memory: likes Python") # insert at a position
last = messages.pop() # remove and return the last item
print(messages)
print("popped:", last)
print("user: hi" in messages)Sorting
list.sort() sorts the list in place and returns None. sorted(items) returns a new sorted list and leaves the original alone. Both accept key= and reverse=True.
scores = [0.41, 0.88, 0.67, 0.91]
print(sorted(scores, reverse=True))
print(scores) # unchanged
words = ["transformer", "rag", "agent"]
words.sort(key=len) # sort by length, in place
print(words)scores = scores.sort() sets scores to None, because sort() works in place and returns nothing. Use either scores.sort() or scores = sorted(scores).
Tuples
An ordered, immutable sequence written with parentheses. Once created it cannot be changed, which makes it a safe container for fixed groups of values such as a (doc_id, score) pair or an (x, y) coordinate.
hit = ("doc-7", 0.93)
doc_id, score = hit # unpacking
print(doc_id, score)
hits = [("doc-1", 0.71), ("doc-7", 0.93)]
for doc_id, score in hits:
print(f"{doc_id}: {score:.0%}")Sets
An unordered collection of unique items. Adding a duplicate does nothing, and checking membership with in is extremely fast, even for millions of items.
tags = ["rag", "llm", "rag", "evals", "llm"]
unique = set(tags)
print(sorted(unique))
seen_ids = {"doc-1", "doc-2"}
print("doc-2" in seen_ids, "doc-9" in seen_ids)
a = {"python", "rag", "agents"}
b = {"rag", "agents", "evals"}
print(sorted(a & b)) # intersection: in both
print(sorted(a | b)) # union: in either
print(sorted(a - b)) # difference: only in aSets have no order, so wrap them in sorted() when you need stable, readable output. Create an empty set with set() — {} creates an empty dict.
Dictionaries
A mutable collection of key → value pairs. Keys are unique and usually strings; values can be anything, including other dicts and lists. Looking up a value by key is very fast. Since Python 3.7, dicts remember insertion order.
A dict is Python's natural representation of a record: one document, one chat message, one evaluation row, one API request body.
doc = {
"id": "doc-1",
"title": "RAG checklist",
"tags": ["retrieval", "eval"],
"score": 0.87,
}
print(doc["title"]) # access by key
print(doc.get("author")) # missing key -> None
print(doc.get("author", "unknown")) # missing key -> default
doc["author"] = "Rahish" # add a key
doc["score"] = 0.9 # update a key
doc.update({"lang": "en", "score": 0.95}) # add/update several keys
del doc["tags"] # remove a key
print(doc)doc["author"] raises KeyError if the key does not exist. When parsing model or API responses, fields are often missing — use .get() with a sensible default, or check with if "author" in doc:.
Looping over a dict
usage = {"prompt_tokens": 812, "completion_tokens": 164}
for key in usage: # keys
print(key)
for key, value in usage.items(): # key/value pairs
print(f"{key:>18}: {value}")
print(sum(usage.values()), "total tokens")Nested data
Real API responses are nested: dicts inside lists inside dicts. Read them from the outside in, one bracket at a time.
response = {
"id": "chatcmpl-123",
"choices": [
{"index": 0, "message": {"role": "assistant", "content": "RAG = retrieval + generation."}}
],
"usage": {"total_tokens": 42},
}
answer = response["choices"][0]["message"]["content"]
print(answer)
print(response["usage"]["total_tokens"])enumerate and zip
enumerate() gives you the index and the item together. zip() walks several lists in parallel, pairing their items.
chunks = ["alpha", "beta", "gamma"]
for i, chunk in enumerate(chunks, start=1):
print(i, chunk)
ids = ["d1", "d2", "d3"]
scores = [0.4, 0.9, 0.7]
pairs = dict(zip(ids, scores))
print(pairs)Comprehensions
A compact expression that builds a new list, dict, or set by transforming and optionally filtering another iterable: [expression for item in items if condition].
Comprehensions replace the very common pattern “create an empty list, loop, append”. They are faster and, when kept short, easier to read.
scores = [0.41, 0.88, 0.67, 0.91]
# the long way
keepers = []
for s in scores:
if s >= 0.7:
keepers.append(s)
print(keepers)
# the comprehension way
print([s for s in scores if s >= 0.7])
# transform every item
print([round(s * 100) for s in scores])
# dict comprehension: id -> word count
docs = {"d1": "vector search basics", "d2": "agents"}
print({doc_id: len(text.split()) for doc_id, text in docs.items()})
# set comprehension: unique first letters
print(sorted({w[0] for w in ["rag", "rerank", "agent"]}))If a comprehension needs nested loops and several conditions, write a normal for loop instead. Readability beats cleverness.
Batching: splitting a list into chunks
Embedding and moderation APIs limit how many inputs you can send per request, so you split your data into batches. This slicing pattern is one you will reuse constantly.
def batched(items: list, size: int) -> list[list]:
return [items[i:i + size] for i in range(0, len(items), size)]
texts = [f"doc {n}" for n in range(1, 8)]
for batch in batched(texts, 3):
print(batch)Python 3.12 added itertools.batched(items, n), which does the same thing lazily and yields tuples.
Helpful tools from collections
Counter counts things; defaultdict creates a default value the first time you touch a missing key, which removes a lot of if key not in d boilerplate.
from collections import Counter, defaultdict
labels = ["correct", "wrong", "correct", "correct", "partial"]
counts = Counter(labels)
print(counts.most_common(2))
by_source = defaultdict(list)
for doc_id, source in [("d1", "wiki"), ("d2", "blog"), ("d3", "wiki")]:
by_source[source].append(doc_id)
print(dict(by_source))JSON
A text format for structured data built from objects ({}), arrays ([]), strings, numbers, booleans, and null. It is the language-independent standard for web APIs — every LLM API request and response is JSON.
Python's built-in json module converts between Python objects and JSON text. Converting to text is called serialising (dumps = “dump string”); converting from text is parsing or deserialising (loads = “load string”).
| Python | JSON |
|---|---|
dict | object {} |
list, tuple | array [] |
str | string (always double quotes) |
int, float | number |
True / False | true / false |
None | null |
import json
payload = {
"model": "gpt-4o-mini",
"messages": [{"role": "user", "content": "Hello"}],
"stream": False,
"stop": None,
}
raw = json.dumps(payload) # Python -> JSON text
print(raw)
back = json.loads(raw) # JSON text -> Python
print(back["messages"][0]["role"], type(back).__name__)Notice how False became false and None became null. For human-readable output use indent, and keep non-English text readable with ensure_ascii=False:
import json
record = {"lang": "hi", "text": "नमस्ते दुनिया", "score": 0.9}
print(json.dumps(record))
print(json.dumps(record, ensure_ascii=False, indent=2))Reading and writing JSON files
json.dump and json.load (no “s”) work directly with open files.
import json
docs = [
{"id": "d1", "text": "Embeddings map text to vectors."},
{"id": "d2", "text": "Rerankers reorder retrieved chunks."},
]
with open("docs.json", "w", encoding="utf-8") as f:
json.dump(docs, f, ensure_ascii=False, indent=2)
with open("docs.json", encoding="utf-8") as f:
loaded = json.load(f)
print(len(loaded), loaded[1]["id"])JSON Lines (JSONL)
JSON Lines: a text file where each line is one complete JSON object. Because records are independent lines, you can append to the file, stream it line by line without loading everything into memory, and recover from one bad line. Fine-tuning datasets, batch API inputs, and evaluation logs all use JSONL.
import json
rows = [
{"id": 1, "question": "What is RAG?", "label": "definition"},
{"id": 2, "question": "Why rerank?", "label": "reasoning"},
]
with open("eval.jsonl", "w", encoding="utf-8") as f:
for row in rows:
f.write(json.dumps(row, ensure_ascii=False) + "\n")
with open("eval.jsonl", encoding="utf-8") as f:
loaded = [json.loads(line) for line in f if line.strip()]
print(open("eval.jsonl", encoding="utf-8").read())
print(loaded[1]["label"])Chat messages: the shape every LLM API uses
Chat models take a list of message dicts. Each message has a role (system, user, or assistant) and content. Keeping the conversation as a plain list makes it trivial to append turns, trim history, and save sessions as JSON.
import json
messages = [
{"role": "system", "content": "You are a concise AI tutor."},
{"role": "user", "content": "What is an embedding?"},
]
messages.append({"role": "assistant", "content": "A vector that captures meaning."})
messages.append({"role": "user", "content": "Give an example use."})
# keep the system prompt plus the last 2 turns to save tokens
trimmed = [messages[0]] + messages[-2:]
print(json.dumps(trimmed, indent=2))Parsing JSON from a model safely
When you ask an LLM to “reply in JSON”, it usually does — but sometimes it wraps the JSON in Markdown fences, adds a sentence, or truncates the output. Always parse defensively.
import json
def parse_model_json(text: str) -> dict | None:
cleaned = text.strip()
if cleaned.startswith("```"):
cleaned = cleaned.strip("`")
cleaned = cleaned.removeprefix("json").strip()
try:
data = json.loads(cleaned)
except json.JSONDecodeError as err:
print("invalid JSON:", err.msg)
return None
if not isinstance(data, dict):
print("expected an object")
return None
return data
print(parse_model_json('{"sentiment": "positive", "confidence": 0.92}'))
print(parse_model_json('```json\n{"sentiment": "negative"}\n```'))
print(parse_model_json('Sure! Here is the JSON: {"sentiment": '))Many model APIs now offer structured outputs or JSON mode, which guarantee syntactically valid JSON that matches a schema you provide. Use it when available — but still validate fields, because valid JSON can still contain wrong values.
Mutability and copying
Assigning a list or dict to a new name does not copy it — both names point to the same object. Changing one changes “both”.
import copy
base = {"model": "gpt-4o-mini", "stop": ["###"]}
alias = base # same object
alias["model"] = "other"
print(base["model"]) # changed too!
base = {"model": "gpt-4o-mini", "stop": ["###"]}
shallow = base.copy() # new dict, but inner list is shared
shallow["stop"].append("END")
print(base["stop"])
base = {"model": "gpt-4o-mini", "stop": ["###"]}
deep = copy.deepcopy(base) # fully independent copy
deep["stop"].append("END")
print(base["stop"], deep["stop"])Use copy.deepcopy() when you build per-request configurations from a shared template, so one request's changes never leak into another.
- Create a list of five document dicts with
id,text, andsource. - Use a comprehension to keep only documents whose text has more than three words.
- Group the ids by
sourcewithdefaultdict(list). - Write the filtered documents to
docs.jsonl, then read the file back and print how many records it contains.
- Lists hold ordered items, tuples hold fixed groups, sets hold unique items, and dicts map keys to values.
- Use
.get(key, default)for fields that might be missing in API or model responses. - Comprehensions,
enumerate,zip, and slicing-based batching cover most data-shaping work. json.dumps/loadswork with strings;json.dump/loadwork with files.- JSONL — one JSON object per line — is the standard format for AI datasets and logs.
- Parse model JSON defensively, and deep-copy shared templates before modifying them.