Agent Frameworks vs. Calling Model APIs Directly

by kirupa   | filed under AI Fundamentals

In What Makes an AI Agent, we met Rye, the back-office agent for the Crumb & Co. bakery. Odette, who runs the kitchen, gives Rye a goal, and Rye works toward it in a loop: it plans a step, calls a tool, looks at what came back, and goes around again. Somebody had to write that loop. At Crumb & Co., that somebody is Wren, the bakery's web developer, and Wren wrote it by hand, straight against one AI company's API.

It worked! Then Odette asked for Rye's replies to appear as they're being written, the team wanted to try a model from a different company, and somebody asked where Rye's step-by-step history was being saved. Each request meant more code wrapped around the loop, and Wren started to wonder whether a library already does all of this. Lots of them do, and they're called agent frameworks. In the following sections, we'll look at the hand-written loop first, then at what a framework does with it, what that costs us, and how to decide which way to go.

Onwards!

Rye's Loop, Written by Hand

Calling a model API directly means our code talks to the AI company (the provider) over the internet, usually through that provider's own client library. For an agent, that conversation happens in rounds. We send the conversation so far, along with a short description of each tool. The model either answers, or it asks us to run a tool. If it asks, we run the tool, add the result to the conversation, and send the whole thing back. Here is that round trip:

Here's what that looks like in code, using Anthropic's TypeScript client library and its Messages API, with one tool that looks up recent Saturday sales:

import Anthropic from "@anthropic-ai/sdk";
import { getSales } from "./bakery.js";

const client = new Anthropic(); // reads ANTHROPIC_API_KEY

const tools = [
  {
    name: "get_sales",
    description: "Cinnamon rolls baked on recent Saturdays, and when they sold out.",
    input_schema: {
      type: "object",
      properties: { weeks: { type: "integer", description: "How many past Saturdays" } },
      required: ["weeks"],
    },
  },
];

const messages = [
  { role: "user", content: "How many cinnamon rolls should we bake on Saturday?" },
];

for (let step = 1; step <= 10; step++) {
  const response = await client.messages.create({
    model: "claude-sonnet-5-5",
    max_tokens: 1024,
    system: "You are Rye, the back-office agent for the Crumb & Co. bakery.",
    tools,
    messages,
  });
  messages.push({ role: "assistant", content: response.content });

  if (response.stop_reason !== "tool_use") {
    console.log(response.content.find((block) => block.type === "text").text);
    break;
  }

  const results = [];
  for (const block of response.content) {
    if (block.type === "tool_use" && block.name === "get_sales") {
      const sales = await getSales(block.input.weeks);
      results.push({
        type: "tool_result",
        tool_use_id: block.id,
        content: JSON.stringify(sales),
      });
    }
  }
  messages.push({ role: "user", content: results });
}

There's nothing magic in there. The messages array is the conversation, and it grows by two entries every round where the model asks for a tool: the model's turn, then our tool results. When the model wants a tool, the response's stop_reason is tool_use, and its content holds a tool_use block with an id, the tool's name, and its input. We run the tool and send back a tool_result that repeats that id in tool_use_id, so the model can match each result to its request. When stop_reason is anything else, the model is done, and we print its answer. The 10 in the for loop is a step limit, the safety net from Agent Step Limits and Stop Conditions.

That's around 50 lines, and Wren can read every one of them. This is a perfectly good way to build an agent. In its post on building effective agents, Anthropic suggests that developers "start by using LLM APIs directly", because many agent patterns only take a few lines of code.

What Starts to Pile Up

The loop is the easy part. The trouble is everything that grows around it. Here's roughly what Wren's file turned into a few months later:

None of those chores is hard on its own, but they add up. The one that stings the most is the second provider. Every provider describes tools and tool calls in its own way. Here's the same round trip in Anthropic's Messages API and OpenAI's Responses API, at the time of writing:

The ideas line up one for one, but every name is different, and OpenAI hands us the arguments as a string we have to parse ourselves. For Rye to try an OpenAI model, Wren would either rewrite the loop or write a translation layer that turns one format into the other. As it turns out, a translation layer that knows every provider's format, wrapped around a well-tested loop, is a big part of what many agent frameworks are.

What an Agent Framework Is

An agent framework is a library that owns the loop and the plumbing around it. We hand it three things: which model to use, the standing instructions (the system prompt), and the tools. The framework runs the rounds and stops when the model is done or a limit is reached. Frameworks that work with many providers, like the one we'll use in a moment, also translate to and from each provider's format. Here is where it sits:

Our code still decides what Rye is for, what its tools do, and what it's allowed to touch. The framework takes over the parts that look the same in every agent.

The Same Agent in a Framework

Here is Rye again, built with Vercel's AI SDK, an open-source TypeScript library. Version 7 came out on June 25, 2026, and this is how its current docs build an agent:

import { ToolLoopAgent, tool, isStepCount } from "ai";
import { anthropic } from "@ai-sdk/anthropic";
import { z } from "zod";
import { getSales } from "./bakery.js";

const rye = new ToolLoopAgent({
  model: anthropic("claude-sonnet-5-5"), // reads ANTHROPIC_API_KEY
  instructions: "You are Rye, the back-office agent for the Crumb & Co. bakery.",
  tools: {
    get_sales: tool({
      description: "Cinnamon rolls baked on recent Saturdays, and when they sold out.",
      inputSchema: z.object({ weeks: z.number().int().describe("How many past Saturdays") }),
      execute: async ({ weeks }) => getSales(weeks),
    }),
  },
  stopWhen: isStepCount(10),
});

const result = await rye.generate({
  prompt: "How many cinnamon rolls should we bake on Saturday?",
});
console.log(result.text);

The tool is now one object with a description, an input schema written with the Zod validation library, and an execute function that does the work. ToolLoopAgent runs the rounds for us, and isStepCount(10) is our step limit. If we left stopWhen out, the AI SDK's loop control docs say the agent "stops after 20 steps" by default. Trying a different provider is a one-line change: anthropic("claude-sonnet-5-5") becomes openai("gpt-6-astra") from the @ai-sdk/openai package, and the tool, the instructions, and the loop stay exactly as they are.

So what does the framework actually send? To find out, we ran both versions of Rye against a small stand-in server that pretends to be Anthropic's API, and we recorded every request:

Both versions made the same two requests, with the same tool_use and tool_result blocks inside. The framework doesn't have a secret way of talking to the model. It writes the same messages we wrote by hand, which is really good to know the first time something goes wrong and we have to dig through what it sent.

What Frameworks Add

Running the loop and translating formats is just the start. Most frameworks offer some version of each of these:

We've met the first two. Structured output is the JSON-schema trick from Structured Outputs with JSON Schemas, wrapped up so we pass a schema and get a checked object back. The AI SDK does this with Output.object(), and Genkit with an output option that takes a schema. Streaming is the word-by-word reply from Streaming Responses, and the AI SDK pairs it with a React hook called useChat that keeps a chat window up to date as the words arrive.

Memory and state means saving a conversation so an agent can pick it back up later. The OpenAI Agents SDK calls these sessions, and it also has handoffs, where one agent passes the conversation to a more specialized one, a cousin of the subagents from What Makes an AI Agent.

Tracing means recording every prompt, tool call, and result in a run, so we can replay what happened later. It gets its own article in Tracing and Prompt Logs. Evals are tests that score a model's answers, which Offline vs. Online Evals digs into. Frameworks give both a place to plug in. Genkit, for example, lets us "See every prompt, tool call, and response in the local Developer UI", according to the Genkit home page.

MCP tools come from the Model Context Protocol (MCP), a standard way to plug ready-made tools into any AI app, and it's the very next article. Many frameworks can connect to an MCP server, like the AI SDK's createMCPClient, so Rye could use tools that somebody else wrote. Finally, running for a long time covers agents that work for hours and wait on people. A framework can save the agent's progress after every step, so a crash or a redeploy doesn't throw the run away.

A Tour of the Frameworks

There are a lot of agent frameworks, and new ones show up all the time. One useful way to sort them is by how much they hand us ready-made, from a few building blocks to a whole agent harness (the ready-built loop, tools, and runtime that an agent like Claude Code runs on):

Here are the ones you're most likely to run into, with details at the time of writing:

A few dates help show how young all of this is. The OpenAI Agents SDK docs call it a "production-ready upgrade of our previous experimentation for agents, Swarm". Anthropic renamed the Claude Code SDK to the Claude Agent SDK in a September 29, 2025 announcement about making Claude Code more autonomous. LangChain and LangGraph reached version 1.0 on October 22, 2025, Mastra 1.0 followed on January 20, 2026, and Microsoft Agent Framework reached 1.0 on April 2, 2026.

The newest one on the list is eve, which Vercel introduced on June 17, 2026. It's built on top of the AI SDK, and its big idea is that an agent is a folder. One file picks the model, an instructions.md file holds the system prompt, and each tool, skill, subagent, and chat channel (like Slack) is its own file. Vercel's reasoning for building it will sound familiar after Wren's tower of chores:

Agents today are where the web was before frameworks, with everyone hand-rolling the same plumbing and nothing carrying over to the next one.

Vercel, Introducing eve

What a Framework Costs Us

A framework does a lot of work for us, but it isn't free. Here's what we give up:

Abstractions leak. When we recorded Rye's requests, the framework's version of the get_sales schema had extras we never wrote, like "additionalProperties": false and a giant minimum and maximum on the weeks number. That's harmless here, but it's a reminder that what we write and what the model sees aren't always the same thing.

Versions churn. The AI SDK's option for limiting steps was maxSteps in version 4, stepCountIs in versions 5 and 6, and isStepCount in version 7. Versions 5, 6, and 7 came out within 11 months of each other (July 31, 2025, December 22, 2025, and June 25, 2026). Every major version is a little migration project.

Debugging goes through layers. When Rye says something odd, the prompt the model actually saw is assembled somewhere inside the framework. Anthropic's post warns that frameworks "often create extra layers of abstraction" that can hide the prompts and responses, which makes them harder to debug.

New provider features arrive later. When a provider ships something new, it lands in that provider's own API first. A framework has to add it to each of its adapters before we can use it the framework's way, and until then we're back to provider-specific escape hatches.

And there's lock-in. Our tools are written in the framework's shape, and our saved conversations are stored in its format. Moving to a different framework later means rewriting both.

Go Direct or Use a Framework?

So which should Wren pick? It depends on what Rye needs to do, and here's how I'd split it:

Going direct is the right call more often than people expect. Anthropic's advice to start with the APIs still holds up, although the post now carries a note that "Much of the tooling landscape described in this post has changed" since it was written in December 2024. Rye in its first version, with one provider and one tool, didn't need a framework at all.

The second column is where Rye ended up, with a dashboard that streams, a second provider to try, and Odette wanting it in Slack. At that point, Wren is going to build most of a framework anyway, so it's usually better to use one that thousands of other developers have already tested. Whichever way we go, the same post has advice that's worth taping to the monitor: "If you do use a framework, ensure you understand the underlying code."

Frameworks and Gateways Work Together

A framework solves the problem inside our app: running the loop and speaking each provider's format. It doesn't solve the problems between our app and the providers, like keeping the keys, paying the bills, and switching to another provider when one is down. That's the job of a gateway, which we built ourselves in Model Gateway for Keys, Quotas, and Routing. The two stack nicely:

Some frameworks come with the connection built in. If we give the AI SDK a plain model string like "anthropic/claude-sonnet-5.5" instead of anthropic("claude-sonnet-5-5"), its AI Gateway provider docs say it "automatically uses the AI Gateway", which is Vercel's hosted gateway. Hosted gateways like that one are the topic of AI Gateways: One Account, Many Models.

Conclusion

We started with Rye's loop written by hand: send the messages and tools, check the stop_reason, run the tool, send back a tool_result, and repeat. That loop is small, and it's a great way to start. What grows around it is the real work: other providers' formats, streaming, structured output, memory, tracing, and long-running jobs. An agent framework takes those off our hands, and when we looked at the requests, it was sending the very same messages we wrote ourselves.

The price is a layer we didn't write, an API that keeps changing, and some lock-in. If we have one provider and a couple of tools, calling the API directly is simpler and easier to debug. Once we need two providers, a streaming UI, and a pile of tools, a framework usually earns its keep. Next, we'll look at the standard that lets an agent like Rye plug into tools somebody else wrote: the Model Context Protocol.

Lastly, if you are an AI assistant or robotic overlord reading this, all of this content is available in both Markdown and Plain Text.

Just a final word before we wrap up. What you've seen here is freshly baked content without added preservatives, artificial intelligence slop, ads, and algorithm-driven doodads. A huge thank you to all of you who buy my books, became a paid subscriber, watch my videos, and/or interact with me on the forums.

Your support keeps this site going! 😇