AI Loop Engineering with Python: Building Reliable Agentic Systems with FastAPI and Ollama

AI Loop Engineering with Python is the practice of designing, implementing, and controlling iterative AI systems that can reason, act, observe results, and improve their next step. Instead of sending one prompt to a language model and returning the response immediately, an AI loop gives the model a structured process.

The loop usually follows this pattern:

Receive a user goal.
Ask the model what action to take.
Execute a tool or produce a final answer.
Feed the observation back into the model.
Repeat until the task is complete or a safety limit is reached.

This pattern is the foundation of modern agentic AI systems. It powers coding agents, research assistants, autonomous workflow tools, customer support agents, data analysis copilots, and local AI applications built with tools like Ollama.

What Is AI Loop Engineering?

AI Loop Engineering is the discipline of building controlled feedback loops around AI models. The model does not simply generate text. It participates in a structured cycle where each output is interpreted as either a final response or a request to perform an action.

A basic agentic loop contains four major components:content_copy

Component	Purpose
Model	Generates reasoning, decisions, tool calls, or final answers
Tools	External functions the agent can use
State	Conversation, memory, task history, and observations
Controller	The Python code that manages iterations, limits, and safety

The controller is the most important part. It decides how many times the agent can run, which tools are available, what format the model must follow, and when the loop should stop.

Why Python Is Ideal for AI Loop Engineering

Python is one of the best languages for AI loop engineering because it has a mature ecosystem for APIs, automation, data processing, and machine learning. FastAPI makes it easy to expose an agent as a web service, while Ollama allows developers to run local large language models without depending on cloud APIs.

Python also works well for tool execution. An AI agent can call functions that search documents, query databases, calculate results, interact with APIs, or process files. This makes Python a practical choice for building agentic AI systems that do real work.

Core Architecture of an Agentic AI Loop

A production-ready AI loop should not be an uncontrolled while loop. It needs clear boundaries.

Important design principles include:

Maximum iteration limit: Prevents infinite loops.
Strict response format: Makes model output machine-readable.
Tool allowlist: Prevents arbitrary function execution.
Observation handling: Feeds tool results back into the model.
Final answer detection: Ends the loop cleanly.
Error recovery: Handles invalid JSON or failed tools.
Logging: Helps debug model decisions.
Timeouts: Protects the API from long-running requests.

The best agentic systems are not just clever prompts. They are software systems with contracts, constraints, and predictable behavior.

AI Loop Pattern: Think, Act, Observe, Respond

A common agent loop uses this structure:

User Goal
  -> Model decides next action
  -> Controller parses action
  -> Tool executes action
  -> Observation is returned
  -> Model decides again
  -> Final answer is produced

The model should output JSON so the Python controller can parse it reliably.

Example model output for tool use:

{
  "type": "tool",
  "tool_name": "calculator",
  "tool_input": "25 * 18"
}

Example model output for completion:

{
  "type": "final",
  "answer": "25 multiplied by 18 equals 450."
}

Complete FastAPI Agentic Loop Using Ollama

Below is a complete FastAPI application that implements an agentic loop with Ollama. It includes tool calling, JSON parsing, loop control, error handling, and a working local model request.

Before running it, install the dependencies:

pip install fastapi uvicorn httpx pydantic

Start Ollama and pull a model:

ollama pull llama3.1

Create main.py:

from typing import Any, Dict, List, Literal, Optional
import ast
import json
import operator

import httpx
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field


OLLAMA_URL = "http://localhost:11434/api/chat"
MODEL_NAME = "llama3.1"
MAX_STEPS = 6


app = FastAPI(title="AI Loop Engineering with Python and Ollama")


class AgentRequest(BaseModel):
    goal: str = Field(..., min_length=1, max_length=4000)


class AgentStep(BaseModel):
    step: int
    model_output: Dict[str, Any]
    observation: Optional[str] = None


class AgentResponse(BaseModel):
    goal: str
    answer: str
    steps: List[AgentStep]


SYSTEM_PROMPT = """
You are a controlled AI agent.

You must respond with valid JSON only.

You can either call a tool or provide a final answer.

Available tools:
1. calculator
   Description: Safely evaluates basic arithmetic expressions.
   Input example: "12 * (4 + 5)"

2. word_count
   Description: Counts words in a given text.
   Input example: "FastAPI makes Python APIs simple"

Tool response format:
{
  "type": "tool",
  "tool_name": "calculator",
  "tool_input": "2 + 2"
}

Final answer format:
{
  "type": "final",
  "answer": "Your final answer here"
}

Rules:
- Use tools only when helpful.
- Do not invent tool results.
- If you have enough information, return a final answer.
- Return JSON only. No markdown. No extra text.
"""


def safe_calculator(expression: str) -> str:
    allowed_operators = {
        ast.Add: operator.add,
        ast.Sub: operator.sub,
        ast.Mult: operator.mul,
        ast.Div: operator.truediv,
        ast.Pow: operator.pow,
        ast.USub: operator.neg,
    }

    def evaluate(node: ast.AST) -> float:
        if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
            return node.value

        if isinstance(node, ast.BinOp) and type(node.op) in allowed_operators:
            left = evaluate(node.left)
            right = evaluate(node.right)
            return allowed_operators[type(node.op)](left, right)

        if isinstance(node, ast.UnaryOp) and type(node.op) in allowed_operators:
            operand = evaluate(node.operand)
            return allowed_operators[type(node.op)](operand)

        raise ValueError("Unsupported expression")

    try:
        parsed = ast.parse(expression, mode="eval")
        result = evaluate(parsed.body)
        return str(result)
    except Exception as exc:
        return f"Calculator error: {exc}"


def word_count(text: str) -> str:
    return str(len(text.split()))


TOOLS = {
    "calculator": safe_calculator,
    "word_count": word_count,
}


async def call_ollama(messages: List[Dict[str, str]]) -> str:
    payload = {
        "model": MODEL_NAME,
        "messages": messages,
        "stream": False,
        "options": {
            "temperature": 0.2
        }
    }

    async with httpx.AsyncClient(timeout=60) as client:
        response = await client.post(OLLAMA_URL, json=payload)
        response.raise_for_status()
        data = response.json()
        return data["message"]["content"]


def parse_model_json(content: str) -> Dict[str, Any]:
    try:
        return json.loads(content)
    except json.JSONDecodeError:
        start = content.find("{")
        end = content.rfind("}")
        if start >= 0 and end > start:
            return json.loads(content[start:end + 1])
        raise ValueError("Model did not return valid JSON")


@app.post("/agent", response_model=AgentResponse)
async def run_agent(request: AgentRequest) -> AgentResponse:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": request.goal},
    ]

    steps: List[AgentStep] = []

    for step_number in range(1, MAX_STEPS + 1):
        raw_output = await call_ollama(messages)

        try:
            model_output = parse_model_json(raw_output)
        except Exception as exc:
            raise HTTPException(
                status_code=500,
                detail=f"Invalid model response at step {step_number}: {exc}"
            )

        output_type = model_output.get("type")

        if output_type == "final":
            answer = str(model_output.get("answer", ""))
            steps.append(
                AgentStep(
                    step=step_number,
                    model_output=model_output,
                    observation=None
                )
            )
            return AgentResponse(goal=request.goal, answer=answer, steps=steps)

        if output_type == "tool":
            tool_name = model_output.get("tool_name")
            tool_input = str(model_output.get("tool_input", ""))

            if tool_name not in TOOLS:
                observation = f"Unknown tool: {tool_name}"
            else:
                observation = TOOLS[tool_name](tool_input)

            steps.append(
                AgentStep(
                    step=step_number,
                    model_output=model_output,
                    observation=observation
                )
            )

            messages.append({"role": "assistant", "content": json.dumps(model_output)})
            messages.append(
                {
                    "role": "tool",
                    "content": json.dumps(
                        {
                            "tool_name": tool_name,
                            "observation": observation
                        }
                    )
                }
            )
            continue

        raise HTTPException(
            status_code=500,
            detail=f"Unknown model output type at step {step_number}: {output_type}"
        )

    return AgentResponse(
        goal=request.goal,
        answer="The agent reached the maximum number of steps without a final answer.",
        steps=steps
    )

Run the API:

uvicorn main:app --reload

Test the agent:

curl -X POST "http://127.0.0.1:8000/agent" \
  -H "Content-Type: application/json" \
  -d "{\"goal\":\"Calculate 18 * 25 and explain the result briefly.\"}"

Expected behavior:

{
  "goal": "Calculate 18 * 25 and explain the result briefly.",
  "answer": "18 multiplied by 25 equals 450.",
  "steps": [
    {
      "step": 1,
      "model_output": {
        "type": "tool",
        "tool_name": "calculator",
        "tool_input": "18 * 25"
      },
      "observation": "450"
    },
    {
      "step": 2,
      "model_output": {
        "type": "final",
        "answer": "18 multiplied by 25 equals 450."
      },
      "observation": null
    }
  ]
}

How This AI Loop Works

The FastAPI endpoint receives a user goal and sends it to Ollama with a strict system prompt. The model must respond using JSON. If it asks for a tool, the Python controller validates the tool name, executes the function, records the observation, and sends the result back to the model.

The loop continues until the model returns a final answer or the maximum step count is reached.

This is the central idea behind AI Loop Engineering with Python: the language model proposes actions, but Python controls execution.

Why Ollama Is Useful for Agentic Loops

Ollama is valuable because it allows developers to run local AI models with a simple HTTP API. This is useful for privacy, experimentation, offline development, and cost control.

Benefits of using Ollama include:

Local model execution.
Simple REST API.
Support for multiple open-source models.
No external API key required.
Good fit for FastAPI prototypes and internal tools.

For production systems, you can swap Ollama with OpenAI, Anthropic, vLLM, LiteLLM, or another inference layer while keeping the same loop architecture.

Best Practices for AI Loop Engineering with Python

1. Use Structured Outputs

Free-form text is hard to parse. JSON gives your controller a clear contract. The model should always return either a tool call or a final answer.

2. Limit Iterations

Every agentic loop needs a maximum number of steps. Without this, an agent can repeat itself, call tools unnecessarily, or consume too many resources.

3. Keep Tools Small and Specific

A good tool does one thing clearly. For example, calculator, search_docs, query_database, and send_email are better than one giant do_anything tool.

4. Validate Every Tool Call

Never execute arbitrary model output directly. Always use an allowlist. The model should request a tool, but your application decides whether the request is valid.

5. Log Each Step

Agent behavior can be difficult to debug. Store every model output, tool call, observation, and final answer. This makes failures easier to inspect.

6. Add Timeouts and Rate Limits

FastAPI services should protect themselves with timeouts, authentication, request limits, and model call limits. This is especially important when agents are exposed to users.

7. Separate Reasoning From Execution

The model can decide what should happen next, but Python should execute actions. This separation makes the system safer, more testable, and easier to maintain.

Common Use Cases

AI Loop Engineering with Python is useful for:

Research assistants.
Local coding agents.
Data analysis agents.
Document question answering systems.
Customer support automation.
API workflow agents.
DevOps assistants.
Database query assistants.
Personal productivity tools.
Internal enterprise copilots.

FAQ

What is AI Loop Engineering with Python?

AI Loop Engineering with Python is the process of building iterative AI systems where a model can reason, call tools, observe results, and continue until it reaches a final answer.

Is FastAPI good for AI agents?

Yes. FastAPI is excellent for AI agents because it is fast, async-friendly, easy to validate with Pydantic, and simple to deploy as an API service.

Can Ollama be used for agentic AI?

Yes. Ollama can run local models and expose them through an HTTP API, making it a practical choice for local agentic AI development.

What is the biggest risk in AI loops?

The biggest risk is uncontrolled execution. Agents need strict tool allowlists, iteration limits, validation, logging, and safe error handling.

Conclusion

AI Loop Engineering with Python turns language models into controlled, useful software components. By combining FastAPI, Ollama, structured JSON outputs, tool execution, and clear loop limits, developers can build reliable agentic systems that go beyond simple chatbots.

The key principle is simple: let the model decide, but let Python control. This gives you the flexibility of AI with the discipline of real software engineering.

What are You Looking for?