Model catalog and deployments

Before you can call a model in Foundry you must deploy it, and the choices you make — which model, which deployment type, how much capacity, which version policy — decide your latency, data residency, reliability, and bill. This lesson explains the catalog, every deployment type, how quotas really work, provisioned throughput, model versions, and the model router.

Last updated: 01 Nov, 2026 · azure-ai-projects 2.x · Python 3.10+

The model catalog

How to choose a model

1. Start from the task. Classification and extraction rarely need the biggest model. Multi-step reasoning, coding, and agents benefit from stronger reasoning models.
2. Check the hard constraints. Context window, modalities (text, image, audio), tool-calling support, structured output support, and region availability.
3. Use the leaderboards in the catalog to compare quality, cost, and throughput side by side — then confirm on your data with an evaluation (lesson 9).
4. Prefer the smallest model that passes your evaluation. It is cheaper, faster, and usually has more available quota.

Deployments: the thing you actually call

You can name a deployment anything. Many teams use the model name (gpt-5-mini) for simplicity, while others use purpose names (chat-default, summariser) so they can swap the underlying model without changing code.

Deployment types

The deployment type controls where inference runs and how you pay. This is one of the most important decisions in this course.

Quota: TPM and RPM

Two details trip people up. First, the limiter estimates each request's cost before it runs, using prompt tokens plus max_output_tokens — so setting a huge maximum output wastes quota even if the reply is short. Second, limits are enforced over short windows (seconds), so bursts can hit 429 even when the per-minute average looks fine.

def capacity(tpm_units: int, rpm_per_1k_tpm: int = 6) -> dict:
    """Standard deployments: capacity is set in units of 1,000 tokens per minute."""
    tpm = tpm_units * 1_000
    return {"TPM": tpm, "RPM": tpm_units * rpm_per_1k_tpm}


def requests_that_fit(tpm: int, avg_in: int, max_out: int) -> int:
    """The limiter counts prompt tokens plus max output tokens per request."""
    return tpm // (avg_in + max_out)


for units in (10, 50, 250):
    cap = capacity(units)
    fit = requests_that_fit(cap["TPM"], avg_in=1_500, max_out=500)
    print(f"{units:>4} units -> {cap['TPM']:>7,} TPM, {cap['RPM']:>5,} RPM, "
          f"~{fit:>4} req/min at 1.5k in + 500 max out")

Notice that a deployment with 60 RPM can only serve about 5 requests a minute of this size — tokens, not requests, are the binding limit. Size capacity from your real prompt sizes and set max_output_tokens realistically.

# deploy a model (check names/versions with: az cognitiveservices account list-models -n $RES -g $RG -o table)
az cognitiveservices account deployment create \
  --name $RES \
  --resource-group $RG \
  --deployment-name gpt-5-mini \
  --model-name gpt-5-mini \
  --model-version "2025-08-07" \
  --model-format OpenAI \
  --sku-name GlobalStandard \
  --sku-capacity 50          # 50 x 1,000 = 50,000 TPM

# see what you have and how much quota is used
az cognitiveservices account deployment list -n $RES -g $RG -o table
az cognitiveservices usage list --location $LOC -o table

Provisioned throughput (PTU)

Use the capacity calculator in the portal to estimate how many PTUs your traffic needs. The cost lesson shows a break-even calculation between the two.

Model versions and upgrades

Models get new versions (for example, a dated snapshot) and old versions are eventually retired. Each deployment has a version-upgrade policy:

Model router

A router is useful for general chat traffic where many requests are simple. For tasks with strict output formats or evaluated prompts, pin a specific model so behaviour stays predictable.

Fine-tuning in one paragraph

Foundry supports supervised fine-tuning, preference fine-tuning (DPO), and reinforcement fine-tuning for selected models. You upload training data as JSONL in chat format, start a job, and deploy the resulting model (often with the cheap Developer type first). Fine-tune only after prompt engineering and RAG are exhausted — it is best for consistent style, format, or narrow domain behaviour, not for teaching new facts.