What is Azure AI Foundry
Export PDFBefore touching code, you need a map. This lesson explains what problem Foundry solves, how it grew out of Azure OpenAI Service, the building blocks you will meet in every later lesson, what you see in the portal, and an honest comparison with the alternatives so you know when Foundry is the right choice.
The problem Foundry solves
Calling a model is easy. Running AI in a company is not. Once an app leaves the prototype stage, a long list of questions appears:
| Question a company asks | Without a platform | With Foundry |
|---|---|---|
| Which models can we use? | Separate contracts and keys per vendor | One catalog, one bill, one endpoint for many vendors |
| Who is allowed to call them? | Shared API keys in config files | Entra ID identities with roles and audit logs |
| Does our data leave our network? | Traffic goes over the public internet | Private endpoints and network isolation |
| Is harmful output blocked? | Each team writes its own filters | Built-in guardrails applied to every call |
| Are the answers any good? | Manual spot checks | Built-in evaluators, datasets, and scheduled evaluation |
| What happened in this bad conversation? | Print statements | End-to-end traces in Application Insights |
| How do we run agents reliably? | Custom servers, queues, and state | A managed Agent Service with conversations and tools |
Foundry is Microsoft's answer: a single platform that handles those concerns so your team can focus on the application logic.
A short history
Knowing the history explains the naming you will see in docs, Stack Overflow answers, and older code.
| When | What happened | What you will see in old material |
|---|---|---|
| 2021–2023 | Azure OpenAI Service launches: OpenAI models hosted inside Azure, managed in Azure OpenAI Studio. | openai.azure.com endpoints, AzureOpenAI client, api-version query strings. |
| Nov 2023 | Azure AI Studio (preview) brings non-OpenAI models, prompt flow, and evaluation together, built on Azure Machine Learning hubs. | Hubs, hub-based projects, prompt flow, Azure ML workspaces. |
| Nov 2024 | Renamed Azure AI Foundry. The Azure AI Agent Service is announced. | “Azure AI Foundry portal”, azure-ai-projects 1.x, create_agent(), threads and runs. |
| 2025 | A new Foundry resource type and Foundry projects replace hubs as the default. One endpoint serves models, agents, and tools. | services.ai.azure.com/api/projects/... endpoints. |
| Nov 2025 | Renamed Microsoft Foundry. A new portal experience and a rebuilt Agent Service on the Responses API. | “Foundry (new)” vs “Foundry (classic)” in the docs. |
| 2026 | azure-ai-projects 2.x goes GA on the v1 REST API. Agents use create_version() and conversations. | This course. |
When you find a tutorial online, check which era it is from. If it uses create_agent(), threads, and runs, it targets the classic Agent Service. If it creates a hub, it targets classic hub-based projects. Both still work but are not where new features land.
The building blocks
The model catalog plus the serving layer. Includes models sold directly by Azure (OpenAI GPT and o-series, Microsoft Phi and MAI, DeepSeek, Meta Llama, Mistral, xAI Grok and others, billed and supported by Microsoft) and models from partners and the community (thousands of open and commercial models, some deployed to managed compute).
A managed runtime for agents. You define an agent — instructions, a model, and tools — and the service runs the reasoning loop, executes built-in tools, stores conversation state, and exposes the agent through an API.
Capabilities agents and apps can use: file search, code interpreter, web search, Azure AI Search, SharePoint, Microsoft Fabric, MCP servers, OpenAPI endpoints, and the classic Azure AI services (Speech, Vision, Language, Translator, Document Intelligence, Content Safety).
Evaluation (scoring quality and safety), red teaming (attacking your own app to find weaknesses), tracing (recording every step of every request), and monitoring dashboards.
Governance features applied across many projects and agents: identity, RBAC, networking, policy, quota, guardrails, and cost visibility.
Tour of the portal
Open ai.azure.com and sign in with your Azure account. Exact labels shift between releases, but you will find these areas:
| Area | What you do there |
|---|---|
| Home / project overview | See the project endpoint, API keys (if enabled), and quick-start code. Copy your endpoint from here. |
| Model catalog (Discover) | Browse and compare models, read model cards, view benchmarks and leaderboards, and deploy. |
| Models + endpoints | Manage deployments: change capacity, version, deployment type, and guardrail. |
| Playgrounds | Chat with a deployment, test images, audio, and agents interactively, then export the code. |
| Agents (Build) | Create and version agents, attach tools and knowledge, and test them. |
| Knowledge / data | Upload files, create datasets, vector stores, and connect search indexes. |
| Evaluation | Run evaluations against datasets or agents, and compare runs. |
| Tracing / monitoring | Inspect traces of agent runs and view dashboards (requires Application Insights). |
| Guardrails + controls | Create content filter policies and blocklists, then assign them to deployments or agents. |
| Management center / admin | Projects, connected resources, users and roles, and quota. |
- Sign in to the portal and find the model catalog.
- Filter by “Models sold directly by Azure” and open the model card for any chat model.
- Note its context window, supported deployment types, and regions. You will use this skill constantly.
How a request flows through Foundry
your app (Python, C#, JS, Java)
│ HTTPS + Entra ID token (or API key)
▼
project endpoint https://<resource>.services.ai.azure.com/api/projects/<project>
│
├─► input guardrail ── blocks harmful prompts and jailbreaks
│
├─► model deployment (gpt-5-mini, DeepSeek-R1, ...) or agent
│ │ agent may call tools: search, code interpreter, MCP, your functions
│
├─► output guardrail ── blocks harmful completions, protected material
│
└─► trace + metrics ──► Application Insights / Azure Monitor
▼
response (text, tool calls, usage, content-filter annotations)Foundry compared with alternatives
| Option | Best when | Trade-off |
|---|---|---|
| Foundry | You are on Azure or Microsoft 365, need enterprise governance, or want many model vendors under one contract. | More concepts to learn than a single API key; some features are Azure-region dependent. |
| Direct vendor API (OpenAI, Anthropic, Google) | Prototypes, startups, or when you need a vendor feature on day one. | You build identity, networking, safety, and evaluation yourself. |
| Azure OpenAI only (classic) | Existing apps that only use OpenAI models. | Still supported, and the same models are available in Foundry; new agent and evaluation features land in Foundry. |
| Azure Machine Learning | Training and serving classic ML or custom deep learning models. | Not focused on generative AI app building. |
| Amazon Bedrock / Google Vertex AI | Your company is standardised on AWS or GCP. | Equivalent ideas with different names; skills transfer well. |
| Self-hosted open models | Full control, offline, or strict data residency. | You operate GPUs, scaling, patching, and safety. |
For models sold directly by Azure, Microsoft states that your prompts and completions are not used to train the underlying models and are not shared with the model provider. Data is processed under Azure's enterprise terms. Always confirm current terms and data residency (for example, with Data Zone deployments) with your compliance team.
Common real-world use cases
- Internal knowledge assistant: employees ask questions over SharePoint, policies, and wikis (RAG + agent).
- Customer support agent: answers from help-center articles, checks order status through an API, and escalates to a human (the project in this course).
- Document processing: extract fields from invoices and contracts with Document Intelligence plus an LLM.
- Developer copilots: code review and test generation over private repositories.
- Contact-center analytics: transcribe calls with Speech, summarise, and score sentiment.
- Foundry is Microsoft's platform for generative AI apps and agents; “Azure AI Foundry” and “Microsoft Foundry” are the same product.
- It adds governance — identity, networking, safety, evaluation, tracing, cost — around the familiar send-messages-get-reply loop.
- The building blocks are Foundry Models, the Agent Service, Foundry Tools, observability, and the control plane.
- Old tutorials use hubs,
create_agent(), and threads; current code uses Foundry projects,create_version(), and conversations. - Choose Foundry when enterprise requirements or multi-vendor models matter; a direct API is fine for quick prototypes.