Curious builder 0 XP earned · 300 to level 2

0 days Finish a lesson to begin

Badge collection

0 of 5 unlocked

small wins to finish your path

Next lesson →

Foundry Agent Service

An agent is a model that can decide what to do next — call a tool, look something up, ask a follow-up — instead of just answering once. The Foundry Agent Service runs that loop for you as a managed, versioned, observable service. This lesson explains the agent types, how versions, conversations, and responses fit together, and shows how to create, run, update, and delete an agent from Python.

Last updated: 01 Nov, 2026 · azure-ai-projects 2.x · Python 3.10+

What is an agent?

Definition · AI agent

A system that uses a language model to pursue a goal over several steps: it reads the conversation, decides whether to answer or use a tool, observes the tool's result, and repeats until it can respond. Instructions define its job; tools define what it can do.

Definition · Foundry Agent Service

Foundry's managed runtime for agents. It stores agent definitions as versions, keeps conversation state, runs the reasoning loop on the Responses API, executes built-in tools (file search, code interpreter, web search, Azure AI Search, MCP, and more) on your behalf, applies guardrails, and emits traces.

Without the service you would write the loop yourself: call the model, parse tool calls, execute them, append results, call again, persist history, handle failures. The Generative AI & Agents course builds that loop by hand; this lesson shows how Foundry manages it.

The agent loop

user message
   │
   ▼
Agent Service ──► model (with instructions + tool definitions + conversation history)
   ▲                  │
   │      ┌───────────┴───────────┐
   │      ▼                       ▼
   │  final answer         tool call(s)
   │      │                 ├─ built-in (file search, code interpreter, web, MCP):
   │      │                 │     executed by the service, result fed back automatically
   │      │                 └─ function tool: returned to YOUR code to execute
   │      ▼                                          │
   │  response to user                               │
   └──────────── function_call_output ◄──────────────┘

Three kinds of agents

Kind	What you provide	Who runs the logic	Use when
Prompt agent	Model + instructions + tools, declared as configuration	The Agent Service	Most single-agent use cases: assistants, support bots, RAG agents
Workflow agent	A declarative graph of steps and agents (built visually or in YAML)	The Agent Service	Multi-step or multi-agent processes with branching, human approval, or handoffs
Hosted agent	Your own agent code in a container (Microsoft Agent Framework, LangGraph, custom)	Your code, hosted and scaled by Foundry	Custom orchestration logic that does not fit configuration

Start with a prompt agent. Move to workflows when you need a defined sequence across several agents, and to hosted agents only when you need full code control.

Core objects

Object	What it is	Lifetime
Agent	A named container, for example `support-agent`	Until deleted
Agent version	An immutable snapshot of the definition (model, instructions, tools). Creating a new version does not change old ones.	Until deleted
Conversation	Stored history of messages and tool calls for one user session	Until deleted
Response	One run of the agent over a conversation, producing output items	Stored with the conversation
Output item	A message, a function call, a tool result, an approval request, or reasoning	Part of a response

Why versions matter

Because versions are immutable, you can test version 4 while production still points at version 3, compare their evaluation scores, and roll back instantly. Treat instructions like code: every change is a new version with a description of what changed.

Create and run an agent

Live Azure call

This example calls your Azure resources, so it needs a deployed project and a signed-in identity. Output will differ on every run.

Python

import os

from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition
from azure.identity import DefaultAzureCredential

project = AIProjectClient(
    endpoint=os.environ["FOUNDRY_PROJECT_ENDPOINT"],
    credential=DefaultAzureCredential(),
)

agent = project.agents.create_version(
    agent_name="azure-tutor",
    definition=PromptAgentDefinition(
        model=os.environ["FOUNDRY_MODEL_DEPLOYMENT"],
        instructions=(
            "You teach Azure AI Foundry to beginners. Explain with one analogy and "
            "end every answer with a one-line summary."
        ),
    ),
    description="v1: initial tutor",
)
print(agent.name, agent.version, agent.id)

azure-tutor 1 azure-tutor:1

Python

agent_ref = {"agent_reference": {"name": agent.name, "type": "agent_reference"}}

with project.get_openai_client() as client:
    conversation = client.conversations.create()

    first = client.responses.create(
        conversation=conversation.id,
        input="What is the difference between a resource and a project?",
        extra_body=agent_ref,
    )
    print(first.output_text, "\n")

    follow_up = client.responses.create(
        conversation=conversation.id,                 # same conversation = memory
        input="And where do model deployments live?",
        extra_body=agent_ref,
    )
    print(follow_up.output_text)

Think of the Foundry resource as an office building and projects as the separate offices inside it. The building provides shared utilities — electricity, security, internet — just as the resource provides model deployments, networking, and billing. Each office keeps its own files and staff, like a project keeps its own agents, data, and evaluations. Summary: the resource is the shared building; projects are private workspaces inside it. Deployments live on the resource — the building — so every project inside can use them and they share the same quota. Summary: deployments belong to the resource and are shared by its projects.

Three details to notice:

You call the agent through the OpenAI-compatible client: conversations and responses live there, not on AIProjectClient.
extra_body={"agent_reference": ...} tells Foundry to run the named agent (its latest version) instead of a raw model. You do not pass model or instructions; the agent version supplies them.
The conversation stores history server-side, so the second call knows what “And” refers to.

Update, inspect, and delete

Python

# a new version — the old one stays available
v2 = project.agents.create_version(
    agent_name="azure-tutor",
    definition=PromptAgentDefinition(
        model=os.environ["FOUNDRY_MODEL_DEPLOYMENT"],
        instructions="You teach Azure AI Foundry. Use one analogy, max 120 words, then a summary line.",
    ),
    description="v2: shorter answers",
)

for version in project.agents.list_versions(agent_name="azure-tutor"):
    print(version.version, version.description)

project.agents.delete_version(agent_name="azure-tutor", agent_version="1")
# project.agents.delete(agent_name="azure-tutor")   # removes the agent and all versions

Agent settings worth knowing

Setting	What it controls
`model`	The deployment name the agent uses.
`instructions`	The system prompt: role, rules, tone, when to use which tool, and what to do when unsure.
`tools`	The tool definitions available to the agent (next lesson).
`tool_choice`	`auto` (model decides), `required`, `none`, or a specific tool.
`temperature` / `top_p`	Sampling randomness, for models that support it (reasoning models often do not).
`reasoning`	Reasoning effort for reasoning models: more effort means better answers on hard tasks, plus more latency and tokens.
`text`	Output format, including JSON schema structured output.
`rai_config`	Which guardrail (content filter) policy applies to the agent.
`metadata` (on the version)	Your own key-value tags: owner, ticket number, evaluation run ID.

Writing good agent instructions

Role and audience: who the agent is and who it serves.
Scope: what it should and should not help with.
Tool rules: when to use each tool, in plain language (“Use get_order_status whenever an order ID is mentioned”).
Grounding rules: answer from tools and knowledge, cite sources, admit uncertainty.
Escalation: when to hand off to a human.
Format: length, tone, structure.

Publishing and consuming agents

Once an agent works, you can expose it beyond your own code: as a dedicated agent endpoint that apps call with the Foundry Agent Consumer role, in Microsoft Teams and Microsoft 365 Copilot, or to other agents through the Agent-to-Agent (A2A) protocol. Publishing features are evolving quickly; check the docs for current options in your region.

Python

# talk to a published agent through its own OpenAI-compatible endpoint (preview)
project = AIProjectClient(endpoint=endpoint, credential=DefaultAzureCredential(), allow_preview=True)
agent_client = project.get_openai_client(agent_name="azure-tutor")

Multi-agent systems and the Agent Framework

Definition · Microsoft Agent Framework

Microsoft's open-source SDK (Python and .NET) for building agents and multi-agent workflows in code — the successor to Semantic Kernel and AutoGen. Its foundry package connects to Foundry models and tools, and its agents can be deployed as Foundry hosted agents.

Common multi-agent patterns are sequential (researcher → writer → reviewer), handoff (a triage agent routes to specialists), group chat (agents take turns under a manager), and magentic (a planner agent delegates dynamically). Use them only when one agent with good tools is genuinely not enough: each extra agent adds latency, cost, and failure modes.

Migrating from the classic Agent Service

Classic (azure-ai-projects 1.x / Assistants)	Current (2.x)
`project.agents.create_agent(model=..., instructions=...)`	`project.agents.create_version(agent_name=..., definition=PromptAgentDefinition(...))`
Threads	Conversations (`client.conversations.create()`)
Messages + runs, polling run status	`client.responses.create(conversation=..., extra_body={"agent_reference": ...})`
Assistants API (`client.beta.assistants`)	Agent versions on the Responses API

Microsoft provides a migration tool for definitions; old thread history is not migrated, so start new conversations after switching.

Try it

Create azure-tutor v1, ask two related questions in one conversation, then ask the second question in a new conversation and compare.
Create v2 with a word limit and confirm the change.
Open the Agents page in the portal and find both versions and the conversation.

Key takeaways

An agent is a model plus instructions plus tools, running a decide-act-observe loop.
The Agent Service manages versions, conversations, the loop, built-in tool execution, guardrails, and traces.
Prompt agents are configuration; workflow agents orchestrate steps; hosted agents run your container code.
Create with project.agents.create_version(); run with client.responses.create(conversation=..., extra_body={"agent_reference": ...}).
Versions are immutable — iterate safely and roll back instantly.