Azure AI Foundry
Export PDFAzure AI Foundry is Microsoft's platform for building generative AI applications and agents on Azure. It puts models from OpenAI, Microsoft, Meta, Mistral, DeepSeek, xAI, Anthropic, and others behind one endpoint, and wraps them with the things a company needs before shipping: identity and access control, private networking, content safety, evaluation, tracing, and cost controls. This course explains every part of the platform, why it exists, and how to use it from Python, ending with a grounded customer-support agent you build end to end.
What this course is about
A unified Azure platform-as-a-service for building, evaluating, deploying, and operating AI applications and agents. It combines a model catalog, a managed agent runtime, tools such as search and code execution, and governance features (identity, networking, safety, monitoring) behind a single resource, portal, and SDK.
Microsoft renamed the product twice in two years. Azure AI Studio became Azure AI Foundry in November 2024, and Azure AI Foundry became Microsoft Foundry in November 2025. The portal lives at ai.azure.com, the docs live under Microsoft Foundry, and many people (and job descriptions) still say “Azure AI Foundry”. They are the same platform. This course uses “Foundry” for short.
If you have built LLM apps with a raw API key, you already know the core loop: send messages, get a reply. Foundry does not change that loop. What it adds is everything around it that an enterprise asks for — who is allowed to call the model, where the data flows, which outputs are blocked, how good the answers are, and what it costs. Knowing these pieces is what separates a weekend demo from a system a bank or hospital will approve.
The one mental model to remember
Azure subscription
└── resource group
└── Foundry resource (kind: AIServices) ── billing, networking, keys, quota
├── model deployments ── gpt-5-mini, DeepSeek-R1, Llama, ...
├── connections ── Azure AI Search, Storage, Bing, MCP servers
└── project(s) ── your workspace, one per app or team
├── agents (versioned) ── instructions + model + tools
├── conversations / responses ── runtime state
├── datasets, vector stores, indexes
├── evaluations, red-team runs
└── traces ──► Application Insights
Your code ──► https://<resource>.services.ai.azure.com/api/projects/<project>
(one endpoint, Entra ID auth, Foundry SDK or OpenAI SDK)Every lesson zooms in on one box of that diagram. By the end you will be able to draw it from memory and explain what each box is for.
What you will learn
| Lesson | What you learn | Why it matters |
|---|---|---|
| What is Azure AI Foundry | History, the building blocks, the portal tour, and when to choose Foundry over alternatives | Gives you the map before the details. |
| Resources, projects, and endpoints | Subscriptions, resource groups, the Foundry resource, projects, regions, hub vs Foundry projects, creating everything with the CLI | Every call, permission, and bill hangs off this hierarchy. |
| Security and access | Entra ID, keyless auth, Foundry RBAC roles, managed identities, keys, Key Vault, private networking | Security review is the first gate in any real deployment. |
| Model catalog and deployments | Models sold by Azure vs partner models, deployment types, quotas (TPM/RPM), provisioned throughput, versions, model router | Wrong deployment choices cause outages and surprise bills. |
| Calling models from code | Responses and Chat Completions, the Foundry SDK and the OpenAI SDK, streaming, structured output, retries | Turns the platform into working code. |
| RAG with Azure AI Search | Indexes, hybrid search, semantic ranking, vector stores, file search, grounding answers with citations | Most enterprise use cases are “answer from our documents”. |
| Foundry Agent Service | Prompt, workflow, and hosted agents; versions; conversations; running agents from code | The managed way to run agents in production. |
| Agent tools | Function calling, MCP servers, Azure AI Search, file search, code interpreter, web grounding, approvals | Tools are what make an agent useful. |
| Evaluation and red teaming | Built-in evaluators, local and cloud evaluation, datasets, quality gates, the AI red teaming agent | You cannot improve what you do not measure. |
| Guardrails and content safety | Content filters, severity levels, Prompt Shields, protected material, groundedness detection, custom policies | Required for responsible and compliant AI. |
| Observability, cost, and operations | Tracing with OpenTelemetry, Application Insights, metrics, quotas, cost levers, CI/CD, infrastructure as code | Keeps the system healthy and affordable after launch. |
| Project: a grounded support agent | An end-to-end build: infrastructure, knowledge, agent, tools, chat loop, evaluation, tracing, cleanup | Ties every lesson together into one portfolio project. |
What you will be able to do
- Create a Foundry resource and project from the portal or the Azure CLI, and explain every field you filled in.
- Authenticate without API keys using Microsoft Entra ID and assign least-privilege roles to people and apps.
- Pick a model and a deployment type deliberately, size its quota, and explain the trade-off between pay-as-you-go and provisioned throughput.
- Call any deployed model from Python with streaming, structured output, and proper retry handling.
- Ground answers in your own documents with Azure AI Search or file search, with citations.
- Build, version, and run agents in the Foundry Agent Service with function tools, MCP tools, and approval steps.
- Evaluate quality and safety with built-in evaluators, and block a release when scores regress.
- Configure guardrails, trace every request, and keep cost under control.
Prerequisites
- Python basics: functions, dicts, JSON, virtual environments (Python for AI Developers).
- LLM basics: tokens, messages, temperature, cost (LLM Fundamentals).
- Helpful: RAG and tool-calling concepts (Building RAG Systems, Generative AI & Agents). Each lesson re-explains what it needs.
- An Azure subscription. A free account works for learning. Some models and deployment types require quota approval in your region.
- No prior Azure experience needed. Every Azure concept (subscription, resource group, RBAC) is defined when it first appears.
Setup for this course
# 1. Azure CLI (2.80+) — macOS shown; see docs for Windows/Linux
brew install azure-cli
az login # opens a browser to sign in
az account show --query name -o tsv # confirm the subscription
# 2. Python environment
python3 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
python -m pip install "azure-ai-projects>=2.3.0" azure-identity openai python-dotenv
python -m pip install azure-ai-evaluation # used in the evaluation lessonLessons mix two kinds of code. Offline examples (parsers, calculators, simulations) run anywhere and show their real output. Live examples call your Azure project; they are marked with a warning box, and their output will differ from run to run.
Foundry itself has no fee, but deployed models, Azure AI Search, and Application Insights bill by usage. Every lesson that creates resources tells you what it costs, and the project lesson ends with a one-line cleanup command. Set a budget alert on your subscription before you start.
Key terms you will meet
| Term | Meaning |
|---|---|
| Subscription | The Azure billing and access boundary. Every resource belongs to exactly one. |
| Resource group | A folder for related resources that share a lifecycle; deleting it deletes everything inside. |
| Foundry resource | The Azure resource (type Microsoft.CognitiveServices/accounts, kind AIServices) that hosts deployments, connections, and projects. |
| Project | A workspace inside a Foundry resource where you build agents, store data, and run evaluations. |
| Deployment | A named, callable instance of a model with its own quota. Your code calls the deployment name, not the model name. |
| Entra ID | Microsoft's identity service (formerly Azure Active Directory). Used for keyless authentication. |
| RBAC | Role-based access control: permissions granted by assigning roles to identities at a scope. |
| TPM / RPM | Tokens per minute and requests per minute — the units of model quota. |
| PTU | Provisioned throughput unit — reserved model capacity billed per hour, not per token. |
| Agent | A versioned definition of instructions, a model, and tools that the Agent Service runs for you. |
| Guardrail / content filter | A policy that scores inputs and outputs for harm and blocks them above a threshold. |
| Evaluator | A function (often an LLM judge) that scores an AI response for quality or safety. |
Azure moves fast. Model names, prices, preview features, and portal layouts change every few months. This course focuses on concepts that stay stable and shows where to check current details. When a specific number appears, treat it as an example and confirm it in the official docs.
Course syllabus
Work through the lessons in order — each builds on the last. Your progress is saved in this browser as you mark lessons done.
Syllabus