How I built a Local AI Agent in C# with Ollama and Microsoft Agent Framework
Build a local AI task assistant using C#, Microsoft Agent Framework, Ollama, and Qwen3:4B — without a cloud AI API.
AI agents are moving beyond simple chatbots.
A chatbot primarily answers questions.
An AI agent can decide what action to take, call application code, maintain conversation state, and return the result.
For .NET developers, this opens up an interesting possibility: building agentic applications using technologies we already know — C#, .NET, and ordinary application methods.
In this tutorial, we'll build a deliberately small but functional AI agent:
A local AI task assistant written in C# that runs on Ollama and can call C# methods as tools.
There will be no database, RAG pipeline, vector database, or multi-agent architecture.
We'll start with the smallest useful agent and build from there.
By the end, you'll have a local task assistant that can understand requests such as:
Add a task called Buy groceries
and decide to call:
AddTask("Buy groceries")
You can then ask:
What tasks do I have?
and the agent can call:
ListTasks()
Or:
Complete task 1
and it can select:
CompleteTask(1)
Everything runs locally.
What Makes This an AI Agent?
This is the key concept we'll explore.
A traditional application might work like this:
Button
↓
AddTask()
The application explicitly decides which function to execute.
An agent works differently:
User request
↓
AI model
↓
Decide which tool is appropriate
↓
C# function
↓
Tool result
↓
AI model
↓
Final response
The model doesn't execute C# code itself.
Instead, it decides which capability provided by our application should be used.
The application executes the function.
This separation between reasoning and execution is one of the fundamental ideas behind tool-using AI agents.
What We're Building
Our architecture will be intentionally simple:
┌─────────────────────────────┐
│ C# Console App │
│ │
│ TaskAssistant │
└──────────────┬──────────────┘
│
▼
┌─────────────────────────────┐
│ Microsoft Agent Framework │
└──────────────┬──────────────┘
│
▼
┌─────────────────────────────┐
│ Ollama │
└──────────────┬──────────────┘
│
▼
┌─────────────────────────────┐
│ Qwen3:4B │
└─────────────────────────────┘
Our agent will have three tools:
AddTask()
ListTasks()
CompleteTask()
The tasks themselves will initially be stored in memory.
That keeps the example focused on the agent architecture, rather than database implementation.
What Is Microsoft Agent Framework?
Microsoft Agent Framework is Microsoft's framework for building AI agents and agentic applications.
The important idea is that you can progressively add capabilities to an agent.
A useful way to think about the learning path is:
First Agent
↓
Tools
↓
Multi-Turn Conversations
↓
Memory & Persistence
↓
Workflows
↓
Hosting
That's also how we'll approach this project.
We won't attempt to build a production-grade multi-agent platform in the first step.
We'll start with:
Model + Instructions + Tools + Session
and then build on top of it.
If you want to understand the architecture behind Microsoft Agent Framework first, you can also read my visual architecture breakdown:
"I Stopped Reading the Microsoft Agent Framework Documentation and Drew This Instead"
Why Use Ollama?
For this project, we'll run the language model locally using Ollama.
This gives us an architecture like:
C# Application
↓
Microsoft Agent Framework
↓
Ollama
↓
Qwen3:4B
The major advantage is that the model runs on your own machine.
You don't need to send prompts to a cloud AI API for this experiment.
That makes this setup particularly useful for:
Learning AI agents
Local development
Experimentation
Prototyping
Privacy-sensitive experiments
Avoiding per-request cloud API costs
For this tutorial, we'll use:
Qwen3:4B
Prerequisites
You'll need:
.NET SDK
Ollama
Qwen3:4B
Visual Studio or VS Code
First verify your .NET installation:
dotnet --version
Then check Ollama:
ollama --version
Pull the model:
ollama pull qwen3:4b
Run it:
ollama run qwen3:4b
Try asking:
What is an AI agent?
If the model responds, your local AI runtime is ready.
Ollama normally exposes its local service at:
http://localhost:11434
Step 1 — Create the C# Project
Create a new console application:
dotnet new console -n TaskAssistant
Move into the project:
cd TaskAssistant
Now install the required packages:
dotnet add package Microsoft.Agents.AI
and:
dotnet add package Microsoft.Extensions.AI.Ollama --prerelease
Note: The Ollama integration package used in this example is a pre-release package. Package versions and APIs can change as Microsoft Agent Framework evolves, so check the current Microsoft documentation if you encounter API differences.
The responsibilities are roughly:
Microsoft.Agents.AI
↓
Agent abstractions and functionality
Microsoft.Extensions.AI.Ollama
↓
Ollama chat client integration
Step 2 — Create the Task Model
Let's start with a simple C# model.
Create a file named:
TaskItem.cs
Add:
public class TaskItem
{
public int Id { get; set; }
public string Title { get; set; } = string.Empty;
public bool Completed { get; set; }
}
For this first version, tasks will exist only in memory.
The lifecycle looks like this:
Application starts
↓
Task list is empty
↓
User adds tasks
↓
Tasks exist in memory
↓
Application exits
↓
Tasks disappear
That's intentional.
We're learning the agent architecture first.
Persistence comes later.
Step 3 — Create the Agent Tools
Now we need to give the agent something useful to do.
Create:
TaskTools.cs
Add:
using System.ComponentModel;
public class TaskTools
{
private readonly List<TaskItem> _tasks = new();
[Description("Add a new task to the user's task list.")]
public string AddTask(
[Description("The title of the task to add.")]
string title)
{
var task = new TaskItem
{
Id = _tasks.Count + 1,
Title = title,
Completed = false
};
_tasks.Add(task);
return $"Task {task.Id} added: {task.Title}";
}
[Description("Return all tasks currently in the user's task list.")]
public List<TaskItem> ListTasks()
{
return _tasks;
}
[Description("Mark an existing task as completed.")]
public string CompleteTask(
[Description("The ID of the task to complete.")]
int id)
{
var task = _tasks.FirstOrDefault(x => x.Id == id);
if (task is null)
{
return $"Task {id} was not found.";
}
task.Completed = true;
return $"Task {id} completed.";
}
}
Notice something important.
There is no AI code inside these methods.
They are ordinary C# methods:
AddTask()
ListTasks()
CompleteTask()
They are simply application capabilities.
The next step is to expose those capabilities to the AI agent.
Step 4 — Turn C# Methods into Agent Tools
Microsoft Agent Framework can expose C# methods as function tools using:
AIFunctionFactory.Create(...)
For example:
AIFunctionFactory.Create(taskTools.AddTask)
Conceptually:
C# Method
↓
AIFunction
↓
Agent Tool
The descriptions we added using Description attributes are also important.
For example:
[Description("Add a new task to the user's task list.")]
helps the model understand what the function does.
Likewise:
[Description("The title of the task to add.")]
helps describe the parameter.
This metadata gives the model information it can use when selecting and calling tools.
Step 5 — Connect Ollama to the Agent
Now we'll create the AI agent.
Open:
Program.cs
Add:
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
Create the Ollama chat client:
var chatClient = new OllamaChatClient(
new Uri("http://localhost:11434"),
"qwen3:4b");
This connects our application to the locally running Ollama service.
Next, create the task tools:
var taskTools = new TaskTools();
Now we can create the agent:
var agent = chatClient.AsAIAgent(
new ChatClientAgentOptions
{
Name = "TaskAssistant",
ChatOptions = new ChatOptions
{
Instructions = """
You are TaskAssistant, an AI task management assistant.
You can:
- Add tasks
- List tasks
- Complete tasks
Always use the appropriate tool when the user
asks to modify or retrieve their tasks.
""",
Tools =
[
AIFunctionFactory.Create(taskTools.AddTask),
AIFunctionFactory.Create(taskTools.ListTasks),
AIFunctionFactory.Create(taskTools.CompleteTask)
]
}
});
This is the most important piece of the application.
We're effectively telling the agent:
You are TaskAssistant.
Here are your instructions.
Here are the capabilities you can use:
AddTask()
ListTasks()
CompleteTask()
The model can then determine which capability is appropriate for a user's request.
Step 6 — Where Does the Agentic Behavior Come From?
Consider this request:
Add a task called Buy groceries
Notice that our application doesn't contain something like:
if (input.Contains("add"))
{
AddTask(...);
}
We're not manually parsing the user's sentence.
Instead, the model knows about the available tools.
Conceptually:
User request
↓
"Add a task called Buy groceries"
↓
AI model
↓
Determines AddTask is appropriate
↓
Microsoft Agent Framework
↓
AddTask("Buy groceries")
↓
C# application
↓
Tool result
↓
AI model
↓
Final response
This is the agentic behavior.
The model makes the tool-selection decision.
The application executes the actual C# function.
The framework orchestrates the interaction between them.
Step 7 — Add a Conversation Session
So far, we've created an agent that can use tools.
Now we want it to support multiple turns.
Create a session:
var session = await agent.CreateSessionAsync();
Then pass that same session when running the agent:
var response = await agent.RunAsync(
input,
session);
Conceptually:
Session
│
┌───────────┼───────────┐
│ │ │
▼ ▼ ▼
Message 1 Message 2 Message 3
Instead of treating every interaction as completely independent, the session provides the context for a multi-turn interaction.
This distinction becomes increasingly important as we move toward more advanced agent applications.
Step 8 — Build the Console Application
Now let's put everything together.
Our complete Program.cs looks like this:
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
var chatClient = new OllamaChatClient(
new Uri("http://localhost:11434"),
"qwen3:4b");
var taskTools = new TaskTools();
var agent = chatClient.AsAIAgent(
new ChatClientAgentOptions
{
Name = "TaskAssistant",
ChatOptions = new ChatOptions
{
Instructions = """
You are TaskAssistant, an AI task management assistant.
You can:
- Add tasks
- List tasks
- Complete tasks
Always use the appropriate tool when the user
asks to modify or retrieve their tasks.
""",
Tools =
[
AIFunctionFactory.Create(taskTools.AddTask),
AIFunctionFactory.Create(taskTools.ListTasks),
AIFunctionFactory.Create(taskTools.CompleteTask)
]
}
});
var session = await agent.CreateSessionAsync();
Console.WriteLine("TaskAssistant");
Console.WriteLine("Type 'exit' to quit.");
Console.WriteLine();
while (true)
{
Console.Write("You: ");
var input = Console.ReadLine();
if (string.IsNullOrWhiteSpace(input))
continue;
if (input.Equals("exit", StringComparison.OrdinalIgnoreCase))
break;
var response = await agent.RunAsync(
input,
session);
Console.WriteLine();
Console.WriteLine($"Agent: {response.Text}");
Console.WriteLine();
}
This is essentially the complete first version of our local AI agent.
Step 9 — Run the Agent
Build the application:
dotnet build
Then run it:
dotnet run
You should see:
TaskAssistant
Type 'exit' to quit.
You:
Now try:
Add a task called Buy groceries
The agent should select:
AddTask()
Try another:
Add a task called Finish my credit card bill payment
Then:
What tasks do I have?
The agent should use:
ListTasks()
Finally:
Complete task 1
The agent should select:
CompleteTask(1)
What Actually Happened?
From the user's perspective, the interaction looks simple:
You:
Add a task called Buy groceries
But internally, the flow is closer to:
User
│
▼
Qwen3:4B
via Ollama
│
▼
Select appropriate tool
│
▼
Microsoft Agent Framework
│
▼
AddTask("Buy groceries")
│
▼
C# application
│
▼
Tool result returned
│
▼
Qwen3:4B
│
▼
Final response
The important distinction is:
The model decides what to do. The application actually does it.
The model does not execute your C# code.
Instead, it requests a tool call.
Your application executes that tool.
The framework connects the two sides.
Why Is This Already an Agent?
At this point, our application has three important characteristics.
1. It Has a Goal
The agent has been instructed to manage tasks.
Task management
2. It Has Capabilities
The agent has access to:
AddTask()
ListTasks()
CompleteTask()
3. It Can Select a Capability
This is the interesting part.
A traditional application might look like:
Button
↓
AddTask()
Button
↓
ListTasks()
Button
↓
CompleteTask()
Our agent instead accepts natural language:
"Please add buying groceries to my task list."
The AI model maps the user's intent to an available capability.
The interface has changed from:
Explicit commands
to:
Natural-language intent
That's one of the simplest ways to understand the difference between a traditional application and an agentic application.
What We Have NOT Built Yet
This is intentionally a small agent.
Our current architecture is:
TaskAssistant
│
┌────────────┼────────────┐
▼ ▼ ▼
AddTask ListTasks CompleteTask
│ │ │
└────────────┼────────────┘
▼
In-Memory List
There are still many things missing.
For example:
Persistent storage
Structured responses
Long-term memory
Error handling
Authentication
API integrations
Observability
Multi-agent workflows
Production hosting
And that's where things start becoming much more interesting.
The first agent doesn't need all of these capabilities.
In fact, adding everything at once often makes it harder to understand what's actually happening.
A better approach is to build the system incrementally.
Session vs Application Data vs Long-Term Memory
One concept becomes particularly important as we move beyond this demo.
These three things are not the same:
Conversation Session
≠
Application Data
≠
Long-Term Agent Memory
Our current session helps maintain the context of the agent interaction.
Our task list is application data.
And long-term memory is a separate concept that we'll need when we want the agent to remember useful information across interactions and potentially across application restarts.
This distinction becomes critical when moving from a simple demo to a production AI application.
What's Next?
We've now built the smallest useful version of our Task Assistant.
It can:
✓ Run a local LLM
✓ Use Microsoft Agent Framework
✓ Understand natural-language requests
✓ Select C# tools
✓ Execute application code
✓ Maintain a conversation session
But there is an obvious problem.
Our tasks exist only in memory.
Restart the application:
dotnet run
and they're gone.
That's our next problem to solve.
The next step isn't another chatbot feature.
It's persistence and memory.
We'll replace the in-memory task list with persistent storage and explore the difference between:
Conversation Session
vs
Application Data
vs
Long-Term Agent Memory
That distinction becomes increasingly important when moving from a local demo agent to a real-world AI application.
Final Architecture
Our first agent may be small, but it contains the fundamental building blocks of an AI agent:
┌───────────────┐
│ User │
└───────┬───────┘
│
▼
┌───────────────┐
│ Qwen3:4B │
│ Ollama │
└───────┬───────┘
│
Tool selection
│
▼
┌─────────────────────────┐
│ Microsoft Agent │
│ Framework │
└────────────┬────────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
AddTask() ListTasks() CompleteTask()
│ │ │
└────────────┼────────────┘
▼
C# Application
│
▼
Task Data
It's small.
But the architecture contains the fundamental building blocks:
Model
+
Instructions
+
Tools
+
State
+
Execution
And that's a much better place to start than jumping directly into a complicated multi-agent architecture.
Conclusion
Building an AI agent doesn't necessarily require a cloud AI API, a vector database, RAG, or a complex multi-agent framework.
With C#, Microsoft Agent Framework, Ollama, and Qwen3:4B, we can build a useful local agent that understands natural language and interacts with ordinary application code.
The most important idea isn't the task application itself.
It's the architecture:
Natural Language
↓
AI Model
↓
Tool Selection
↓
C# Function
↓
Application Result
↓
AI Response
Once you understand this loop, you can start replacing the simple task functions with real application capabilities:
CreateOrder()
GetCustomer()
SearchDocuments()
SendEmail()
CreateTicket()
QueryDatabase()
CallExternalAPI()
That's where AI agents become much more than chatbots.
If you'd like the complete working project, leave a comment with "Repo" and I'll share it.
More Microsoft Agent Framework
If you're learning Microsoft's agent ecosystem, check out my related article:
I Stopped Reading the Microsoft Agent Framework Documentation and Drew This Instead link
It provides a visual explanation of the architecture before diving into implementation.
Comments
Post a Comment