# Agent Frameworks vs. Calling Model APIs Directly

by [kirupa](https://www.kirupa.com/me/index.htm) | filed under [AI Fundamentals](https://www.kirupa.com/ai_fundamentals/index.htm)

In [What Makes an AI Agent](https://www.kirupa.com/ai_fundamentals/ai_agents.htm), we met Rye, the back-office agent for the Crumb & Co. bakery. Odette, who runs the kitchen, gives Rye a goal, and Rye works toward it in a loop: it plans a step, calls a tool, looks at what came back, and goes around again. Somebody had to write that loop. At Crumb & Co., that somebody is Wren, the bakery's web developer, and Wren wrote it by hand, straight against one AI company's API.

It worked! Then Odette asked for Rye's replies to appear as they're being written, the team wanted to try a model from a different company, and somebody asked where Rye's step-by-step history was being saved. Each request meant more code wrapped around the loop, and Wren started to wonder whether a library already does all of this. Lots of them do, and they're called **agent frameworks**. In the following sections, we'll look at the hand-written loop first, then at what a framework does with it, what that costs us, and how to decide which way to go.

Onwards!

## Rye's Loop, Written by Hand

Calling a model API directly means our code talks to the AI company (the **provider**) over the internet, usually through that provider's own client library. For an agent, that conversation happens in rounds. We send the conversation so far, along with a short description of each tool. The model either answers, or it asks us to run a tool. If it asks, we run the tool, add the result to the conversation, and send the whole thing back. Here is that round trip:

![Rye's hand-written loop has four steps. 1, send the conversation, which is the messages plus the tools. 2, the model replies, and we check its stop_reason. If it asks for a tool, 3, we run the tool, such as get_sales with 2, and 4, we add a tool_result to the messages and go around again, for up to 10 rounds. If the model answers instead, we're done and print the answer.](https://www.kirupa.com/ai_fundamentals/images/agent_frameworks_direct_loop.png)

Here's what that looks like in code, using Anthropic's TypeScript client library and its Messages API, with one tool that looks up recent Saturday sales:

```js
import Anthropic from "@anthropic-ai/sdk";
import { getSales } from "./bakery.js";

const client = new Anthropic(); // reads ANTHROPIC_API_KEY

const tools = [
  {
    name: "get_sales",
    description: "Cinnamon rolls baked on recent Saturdays, and when they sold out.",
    input_schema: {
      type: "object",
      properties: { weeks: { type: "integer", description: "How many past Saturdays" } },
      required: ["weeks"],
    },
  },
];

const messages = [
  { role: "user", content: "How many cinnamon rolls should we bake on Saturday?" },
];

for (let step = 1; step <= 10; step++) {
  const response = await client.messages.create({
    model: "claude-sonnet-5-5",
    max_tokens: 1024,
    system: "You are Rye, the back-office agent for the Crumb & Co. bakery.",
    tools,
    messages,
  });
  messages.push({ role: "assistant", content: response.content });

  if (response.stop_reason !== "tool_use") {
    console.log(response.content.find((block) => block.type === "text").text);
    break;
  }

  const results = [];
  for (const block of response.content) {
    if (block.type === "tool_use" && block.name === "get_sales") {
      const sales = await getSales(block.input.weeks);
      results.push({
        type: "tool_result",
        tool_use_id: block.id,
        content: JSON.stringify(sales),
      });
    }
  }
  messages.push({ role: "user", content: results });
}
```

There's nothing magic in there. The `messages` array is the conversation, and it grows by two entries every round where the model asks for a tool: the model's turn, then our tool results. When the model wants a tool, the response's `stop_reason` is `tool_use`, and its content holds a `tool_use` block with an `id`, the tool's name, and its input. We run the tool and send back a `tool_result` that repeats that id in `tool_use_id`, so the model can match each result to its request. When `stop_reason` is anything else, the model is done, and we print its answer. The `10` in the `for` loop is a step limit, the safety net from [Agent Step Limits and Stop Conditions](https://www.kirupa.com/ai_fundamentals/agent_step_limits.htm).

That's around 50 lines, and Wren can read every one of them. This is a perfectly good way to build an agent. In its post on [building effective agents](https://www.anthropic.com/research/building-effective-agents), Anthropic suggests that developers "start by using LLM APIs directly", because many agent patterns only take a few lines of code.

## What Starts to Pile Up

The loop is the easy part. The trouble is everything that grows around it. Here's roughly what Wren's file turned into a few months later:

![A tower of chores stacked on top of a small block labeled the tool loop. The chores are a second provider's format, streaming replies to the dashboard, turning replies into JSON, saving each conversation, retrying when the API hiccups, logging every step, and asking Odette before changing the bake list. A handwritten note points at the bottom block: the loop is the small part!](https://www.kirupa.com/ai_fundamentals/images/agent_frameworks_pile_up.png)

None of those chores is hard on its own, but they add up. The one that stings the most is the second provider. Every provider describes tools and tool calls in its own way. Here's the same round trip in Anthropic's [Messages API](https://platform.claude.com/docs/en/agents-and-tools/tool-use/handle-tool-calls) and OpenAI's [Responses API](https://developers.openai.com/api/docs/guides/function-calling), at the time of writing:

| Step | Anthropic Messages API | OpenAI Responses API |
|---|---|---|
| Standing instructions go in | `system` | `instructions` |
| A tool's inputs are described by | `input_schema` | `parameters` |
| The model asks for a tool with | a `tool_use` block | a `function_call` item |
| The tool's arguments arrive as | an object, in `input` | a JSON string, in `arguments` |
| We send the result back as | `tool_result`, matched by `tool_use_id` | `function_call_output`, matched by `call_id` |

The ideas line up one for one, but every name is different, and OpenAI hands us the arguments as a string we have to parse ourselves. For Rye to try an OpenAI model, Wren would either rewrite the loop or write a translation layer that turns one format into the other. As it turns out, a translation layer that knows every provider's format, wrapped around a well-tested loop, is a big part of what many agent frameworks are.

## What an Agent Framework Is

An **agent framework** is a library that owns the loop and the plumbing around it. We hand it three things: which model to use, the standing instructions (the [system prompt](https://www.kirupa.com/ai_fundamentals/system_prompts_vs_user_prompts.htm)), and the tools. The framework runs the rounds and stops when the model is done or a limit is reached. Frameworks that work with many providers, like the one we'll use in a moment, also translate to and from each provider's format. Here is where it sits:

![Four layers. On top, our app holds Rye's instructions, its tools, and what it is allowed to touch. Below it, the agent framework runs the loop, enforces step limits, streams replies, keeps memory, and records traces. Below that, provider adapters translate for Anthropic, OpenAI, and Google. At the bottom are the model APIs themselves.](https://www.kirupa.com/ai_fundamentals/images/agent_frameworks_layers.png)

Our code still decides what Rye is for, what its tools do, and what it's allowed to touch. The framework takes over the parts that look the same in every agent.

## The Same Agent in a Framework

Here is Rye again, built with Vercel's [AI SDK](https://ai-sdk.dev/docs/agents/overview), an open-source TypeScript library. Version 7 came out on June 25, 2026, and this is how its current docs build an agent:

```js
import { ToolLoopAgent, tool, isStepCount } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { z } from "zod";
import { getSales } from "./bakery.js";

const rye = new ToolLoopAgent({
  model: anthropic("claude-sonnet-5-5"), // reads ANTHROPIC_API_KEY
  instructions: "You are Rye, the back-office agent for the Crumb & Co. bakery.",
  tools: {
    get_sales: tool({
      description: "Cinnamon rolls baked on recent Saturdays, and when they sold out.",
      inputSchema: z.object({ weeks: z.number().int().describe("How many past Saturdays") }),
      execute: async ({ weeks }) => getSales(weeks),
    }),
  },
  stopWhen: isStepCount(10),
});

const result = await rye.generate({
  prompt: "How many cinnamon rolls should we bake on Saturday?",
});
console.log(result.text);
```

The tool is now one object with a description, an input schema written with the Zod validation library, and an `execute` function that does the work. `ToolLoopAgent` runs the rounds for us, and `isStepCount(10)` is our step limit. If we left `stopWhen` out, the AI SDK's [loop control docs](https://ai-sdk.dev/docs/agents/loop-control) say the agent "stops after 20 steps" by default. Trying a different provider is a one-line change: `anthropic("claude-sonnet-5-5")` becomes `openai("gpt-6-astra")` from the `@ai-sdk/openai` package, and the tool, the instructions, and the loop stay exactly as they are.

So what does the framework actually send? To find out, we ran both versions of Rye against a small stand-in server that pretends to be Anthropic's API, and we recorded every request:

![Two versions of Rye, our own loop and the AI SDK's ToolLoopAgent, both send the same two requests. Request 1 sends the messages and tools, and the model answers with tool_use for get_sales. Request 2 adds the tool_result, and the model answers with the final text.](https://www.kirupa.com/ai_fundamentals/images/agent_frameworks_same_wire.png)

Both versions made the same two requests, with the same `tool_use` and `tool_result` blocks inside. The framework doesn't have a secret way of talking to the model. It writes the same messages we wrote by hand, which is really good to know the first time something goes wrong and we have to dig through what it sent.

## What Frameworks Add

Running the loop and translating formats is just the start. Most frameworks offer some version of each of these:

![Nine tiles in a framework's toolbox: one model interface, the loop with step limits, structured output, streaming to the UI, memory and state, handoffs between agents, tracing and evals, MCP tools, and running for a long time.](https://www.kirupa.com/ai_fundamentals/images/agent_frameworks_toolbox.png)

We've met the first two. **Structured output** is the JSON-schema trick from [Structured Outputs with JSON Schemas](https://www.kirupa.com/ai_fundamentals/structured_outputs_json_schema.htm), wrapped up so we pass a schema and get a checked object back. The AI SDK does this with `Output.object()`, and Genkit with an `output` option that takes a schema. **Streaming** is the word-by-word reply from [Streaming Responses](https://www.kirupa.com/ai_fundamentals/streaming_responses.htm), and the AI SDK pairs it with a React hook called `useChat` that keeps a chat window up to date as the words arrive.

**Memory and state** means saving a conversation so an agent can pick it back up later. The [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) calls these sessions, and it also has **handoffs**, where one agent passes the conversation to a more specialized one, a cousin of the subagents from What Makes an AI Agent.

**Tracing** means recording every prompt, tool call, and result in a run, so we can replay what happened later. It gets its own article in [Tracing and Prompt Logs](https://www.kirupa.com/ai_fundamentals/tracing_and_prompt_logs.htm). **Evals** are tests that score a model's answers, which [Offline vs. Online Evals](https://www.kirupa.com/ai_fundamentals/offline_vs_online_evals.htm) digs into. Frameworks give both a place to plug in. Genkit, for example, lets us "See every prompt, tool call, and response in the local Developer UI", according to the [Genkit home page](https://genkit.dev/).

**MCP tools** come from the [Model Context Protocol (MCP)](https://www.kirupa.com/ai_fundamentals/model_context_protocol.htm), a standard way to plug ready-made tools into any AI app, and it's the very next article. Many frameworks can connect to an MCP server, like the AI SDK's `createMCPClient`, so Rye could use tools that somebody else wrote. Finally, **running for a long time** covers agents that work for hours and wait on people. A framework can save the agent's progress after every step, so a crash or a redeploy doesn't throw the run away.

## A Tour of the Frameworks

There are a lot of agent frameworks, and new ones show up all the time. One useful way to sort them is by how much they hand us ready-made, from a few building blocks to a whole agent harness (the ready-built loop, tools, and runtime that an agent like Claude Code runs on):

![Agent frameworks along a line from building blocks we assemble to harnesses and runtimes where more comes built in. Building blocks: the AI SDK and Genkit. Agent frameworks: the OpenAI Agents SDK, LangChain and LangGraph, Mastra, CrewAI, LlamaIndex, Microsoft Agent Framework, and Google ADK. Harnesses and runtimes: the Claude Agent SDK and eve.](https://www.kirupa.com/ai_fundamentals/images/agent_frameworks_spectrum.png)

Here are the ones you're most likely to run into, with details at the time of writing:

| Framework | From | Languages | Known for |
|---|---|---|---|
| [AI SDK](https://ai-sdk.dev/docs/introduction) | Vercel | TypeScript | One model interface, `ToolLoopAgent`, and streaming chat UIs |
| [eve](https://vercel.com/blog/introducing-eve) | Vercel | TypeScript | An agent defined as a folder of files, with durable sessions, approvals, and sandboxes (walled-off places to run code) built in |
| [Genkit](https://genkit.dev/) | Google | TypeScript, Go, and Dart, with Python in preview | Flows, model plugins, and a local Developer UI for tracing |
| [Agent Development Kit (ADK)](https://adk.dev/) | Google | Python, TypeScript, Go, Java, and Kotlin | Building and deploying agents at enterprise scale |
| [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) | OpenAI | Python and TypeScript | A small set of pieces: agents, handoffs, guardrails, sessions, and tracing |
| [Claude Agent SDK](https://code.claude.com/docs/en/agent-sdk/overview) | Anthropic | Python and TypeScript | The same tools, agent loop, and context management that power Claude Code |
| [LangChain and LangGraph](https://www.langchain.com/blog/langchain-langgraph-1dot0) | LangChain | Python and TypeScript | `create_agent`, plus LangGraph for long-running, stateful agents |
| [Mastra](https://mastra.ai/blog/announcing-mastra-1) | Mastra | TypeScript | Agents, workflows, and memory in one package |
| [LlamaIndex](https://developers.llamaindex.ai/python/framework/) | LlamaIndex | Python | Agents that work over your own documents and data |
| [CrewAI](https://docs.crewai.com/) | CrewAI | Python | Crews of agents that work as a team, steered by flows |
| [Microsoft Agent Framework](https://devblogs.microsoft.com/agent-framework/microsoft-agent-framework-at-build-2026-announce/) | Microsoft | .NET and Python | Brings AutoGen and Semantic Kernel together in one supported framework |

A few dates help show how young all of this is. The [OpenAI Agents SDK docs](https://openai.github.io/openai-agents-python/) call it a "production-ready upgrade of our previous experimentation for agents, Swarm". Anthropic renamed the Claude Code SDK to the Claude Agent SDK in a [September 29, 2025 announcement](https://www.anthropic.com/news/enabling-claude-code-to-work-more-autonomously) about making Claude Code more autonomous. LangChain and LangGraph reached version 1.0 on October 22, 2025, Mastra 1.0 followed on January 20, 2026, and Microsoft Agent Framework reached 1.0 on April 2, 2026.

The newest one on the list is eve, which Vercel introduced on June 17, 2026. It's built on top of the AI SDK, and its big idea is that an agent is a folder. One file picks the model, an `instructions.md` file holds the system prompt, and each tool, skill, subagent, and chat channel (like Slack) is its own file. Vercel's reasoning for building it will sound familiar after Wren's tower of chores:

> Agents today are where the web was before frameworks, with everyone hand-rolling the same plumbing and nothing carrying over to the next one.
>
> [Vercel, Introducing eve](https://vercel.com/blog/introducing-eve)

## What a Framework Costs Us

A framework does a lot of work for us, but it isn't free. Here's what we give up:

![Five costs of a framework. Abstraction leaks: it sends schema rules we never wrote. Version churn: the step limit went from maxSteps to stepCountIs to isStepCount. Layers to debug: the real prompt is a few files away. Lagging features: a provider's newest feature arrives later. Lock-in: our tools and saved state take its shape.](https://www.kirupa.com/ai_fundamentals/images/agent_frameworks_costs.png)

**Abstractions leak.** When we recorded Rye's requests, the framework's version of the `get_sales` schema had extras we never wrote, like `"additionalProperties": false` and a giant minimum and maximum on the `weeks` number. That's harmless here, but it's a reminder that what we write and what the model sees aren't always the same thing.

**Versions churn.** The AI SDK's option for limiting steps was `maxSteps` in version 4, `stepCountIs` in versions 5 and 6, and `isStepCount` in version 7. Versions 5, 6, and 7 came out within 11 months of each other (July 31, 2025, December 22, 2025, and June 25, 2026). Every major version is a little migration project.

**Debugging goes through layers.** When Rye says something odd, the prompt the model actually saw is assembled somewhere inside the framework. Anthropic's post warns that frameworks "often create extra layers of abstraction" that can hide the prompts and responses, which makes them harder to debug.

**New provider features arrive later.** When a provider ships something new, it lands in that provider's own API first. A framework has to add it to each of its adapters before we can use it the framework's way, and until then we're back to provider-specific escape hatches.

**And there's lock-in.** Our tools are written in the framework's shape, and our saved conversations are stored in its format. Moving to a different framework later means rewriting both.

## Go Direct or Use a Framework?

So which should Wren pick? It depends on what Rye needs to do, and here's how I'd split it:

![Two columns. Go direct when we use one provider, have a handful of tools, run a short simple loop, want to see every byte, or are learning how agents work. Reach for a framework when we use more than one provider, stream replies to a UI, plug in many tools or MCP servers, have agents that hand off work, or run jobs that take hours.](https://www.kirupa.com/ai_fundamentals/images/agent_frameworks_decision.png)

Going direct is the right call more often than people expect. Anthropic's advice to start with the APIs still holds up, although the post now carries a note that "Much of the tooling landscape described in this post has changed" since it was written in December 2024. Rye in its first version, with one provider and one tool, didn't need a framework at all.

The second column is where Rye ended up, with a dashboard that streams, a second provider to try, and Odette wanting it in Slack. At that point, Wren is going to build most of a framework anyway, so it's usually better to use one that thousands of other developers have already tested. Whichever way we go, the same post has advice that's worth taping to the monitor: "If you do use a framework, ensure you understand the underlying code."

## Frameworks and Gateways Work Together

A framework solves the problem inside our app: running the loop and speaking each provider's format. It doesn't solve the problems between our app and the providers, like keeping the keys, paying the bills, and switching to another provider when one is down. That's the job of a gateway, which we built ourselves in [Model Gateway for Keys, Quotas, and Routing](https://www.kirupa.com/ai_fundamentals/model_gateway.htm). The two stack nicely:

![Rye's app contains the agent framework, which runs the loop and speaks one model interface. The app sends every request to an AI gateway, which holds one key, one bill, and the failover rules, and the gateway forwards each request to Anthropic, OpenAI, or Google.](https://www.kirupa.com/ai_fundamentals/images/agent_frameworks_with_gateway.png)

Some frameworks come with the connection built in. If we give the AI SDK a plain model string like `"anthropic/claude-sonnet-5.5"` instead of `anthropic("claude-sonnet-5-5")`, its [AI Gateway provider docs](https://ai-sdk.dev/providers/ai-sdk-providers/ai-gateway) say it "automatically uses the AI Gateway", which is Vercel's hosted gateway. Hosted gateways like that one are the topic of [AI Gateways: One Account, Many Models](https://www.kirupa.com/ai_fundamentals/ai_gateways.htm).

## Conclusion

We started with Rye's loop written by hand: send the messages and tools, check the `stop_reason`, run the tool, send back a `tool_result`, and repeat. That loop is small, and it's a great way to start. What grows around it is the real work: other providers' formats, streaming, structured output, memory, tracing, and long-running jobs. An agent framework takes those off our hands, and when we looked at the requests, it was sending the very same messages we wrote ourselves.

The price is a layer we didn't write, an API that keeps changing, and some lock-in. If we have one provider and a couple of tools, calling the API directly is simpler and easier to debug. Once we need two providers, a streaming UI, and a pile of tools, a framework usually earns its keep. Next, we'll look at the standard that lets an agent like Rye plug into tools somebody else wrote: the Model Context Protocol.