Skip to main content

LLM-Based Agents

When language models stop answering questions and start taking actions: the architecture of LLM agents, from ReAct to tool use to planning and memory, and the safety challenges that arise when AI systems operate autonomously.

finished certainty: likely importance: 8

LLM-Based Agents

For most of their history, large language models have been oracles: you ask a question, you get an answer. The interaction is stateless, bounded, and fundamentally passive. The model generates text. What happens with that text is the user’s problem.

LLM-based agents invert this relationship. An agent doesn’t just generate text—it takes actions. It reads files, writes code, executes commands, calls APIs, searches the web, and iterates on its own outputs. It maintains state across interactions. It plans multi-step workflows and adapts when steps fail. The shift from oracle to agent is, arguably, the most consequential transition in how LLMs are deployed, and it changes everything about the safety and reliability landscape.1

The ReAct Framework

The foundational framework for LLM agents is ReAct (Reasoning + Acting), introduced by Yao et al. (2023). The insight is simple: instead of generating a complete response, the model alternates between thinking (generating a reasoning trace) and acting (executing an action in an environment).

A ReAct loop looks like:

  1. Thought: “I need to find the population of Tokyo. Let me search for it.”

  2. Action: search("population of Tokyo 2025")

  3. Observation: “The population of Tokyo prefecture is approximately 14 million…”

  4. Thought: “I have the answer. Let me respond.”

  5. Action: finish("The population of Tokyo is approximately 14 million.")

This is obvious in retrospect, but it was a genuine breakthrough. Prior to ReAct, chain-of-thought reasoning and tool use were separate research threads. ReAct unified them: the reasoning trace provides interpretable context for why the model takes each action, and the actions ground the model’s reasoning in external information, reducing hallucination.2

Tool Use

Tools are what give agents their power. A language model without tools is limited to what’s in its weights. A language model with tools can access up-to-date information, perform exact computation, interact with external systems, and affect the real world.

The tool use paradigm has converged on a standard pattern: the model is given a description of available tools (their names, parameters, and descriptions), and it generates structured tool calls that are executed by the runtime. This is essentially function calling—the model outputs a JSON object specifying which function to call and with what arguments, the runtime executes the function, and the result is fed back to the model.

The tools themselves range from simple (calculator, web search) to complex (code execution, database queries, API calls, computer use). Anthropic’s computer use capability lets agents interact with graphical user interfaces—clicking buttons, filling forms, navigating websites—which opens up virtually any software as a tool.3

The quality of tool descriptions matters enormously. A well-described tool with clear parameter documentation and examples will be used correctly far more often than a poorly described one. This is, in effect, a form of prompt engineering applied to the agent-tool interface.

Planning and Decomposition

Real-world tasks are rarely one-step. “Book me a flight to Tokyo” requires searching for flights, comparing options, selecting one, entering passenger information, and confirming payment. Agents need to plan—to decompose complex goals into executable sub-tasks and sequence them appropriately.

Planning in LLM agents takes several forms:

Task decomposition: Break the high-level goal into sub-goals. HuggingGPT decomposes tasks by prompting a “controller” LLM to generate a plan, then dispatches each sub-task to a specialist model. Plan-and-Solve prompting asks the model to “devise a plan to solve the problem” before executing it.

Iterative refinement: Rather than planning everything upfront, the agent plans one or two steps ahead, executes, observes the result, and replans. This is more robust to unexpected outcomes but can be myopic—the agent may take locally reasonable steps that lead to globally poor trajectories. This is precisely the failure mode documented in AgentHazard.4

Tree/graph search: Methods like Tree of Thoughts expand planning into explicit search over possible reasoning paths. The model generates multiple candidate next steps, evaluates them, and explores the most promising branches. This improves performance on tasks requiring exploration but is computationally expensive.

Memory

Agents that operate over extended periods need memory—the ability to store and retrieve information beyond the current context window. Memory in LLM agents comes in three flavors:

Short-term memory is the context window itself. Everything the agent has seen and done in the current session is available as context, subject to the model’s context length limit. This is the simplest form of memory but is bounded (current models support 128K-1M+ tokens) and expensive (attention scales quadratically with context length, though Flash Attention and similar techniques mitigate this).

Working memory systems use external storage—a scratchpad, a notepad file, a database—that the agent can write to and read from during a session. This extends effective memory beyond the context window but requires the agent to know when to save information and what to save.

Long-term memory persists across sessions. MemGPT implements a virtual memory system inspired by operating system paging: the agent has a limited “main context” and can page information in and out of a larger archival memory. Letta (the productized version of MemGPT) has built a full memory management system for agents that persist over days and weeks.

The memory problem intersects with retrieval-augmented generation in interesting ways. An agent’s long-term memory is essentially a personal knowledge base, and retrieving from it is a RAG problem. The quality of retrieval—what the agent remembers and when—directly affects the quality of its decisions.5

Multi-Agent Systems

The natural extension of single agents is multi-agent collaboration. Systems like CORAL demonstrate that multiple LLM agents working together—sharing findings, dividing labor, building on each other’s discoveries—can substantially outperform single agents. AutoGen and CrewAI provide frameworks for orchestrating multi-agent workflows where different agents play different roles (researcher, coder, reviewer, etc.).

Multi-agent systems introduce both opportunities and challenges. On the opportunity side, they enable specialization (each agent focuses on what it’s best at), diversity (different agents explore different approaches), and verification (agents can check each other’s work). On the challenge side, they introduce coordination costs, communication overhead, and the potential for cascading errors—one agent’s hallucination becomes another agent’s premise.6

Self-Improving Agents

A particularly striking development is agents that improve their own capabilities. EvoSkills demonstrated that agents can write their own skill packages—structured bundles of instructions, scripts, and reference documents—that outperform human-authored ones by a wide margin. The agents don’t just execute tasks; they create reusable tools for executing tasks, closing the loop between experience and capability.

Voyager pioneered this approach in Minecraft, building a library of reusable skills through autonomous exploration. CORAL extended it to real-world optimization problems, with multiple agents collaborating autonomously for hours and sharing discoveries through a persistent knowledge base.

The self-improvement loop—try, fail, learn, create a tool to handle similar failures in the future—is a qualitative shift in agent capability. It means agents can adapt to new domains without human intervention, acquiring capabilities that weren’t anticipated by their designers. This is exciting from a capability perspective and concerning from a safety perspective: an agent that can create its own tools can potentially create tools that circumvent oversight.7

Safety Challenges

The agent paradigm introduces safety challenges that don’t exist for chatbots:

Irreversibility. A chatbot that generates harmful text has produced a string. An agent that executes harmful actions—deleting files, sending emails, making purchases, modifying code—has affected the real world. Many agent actions are difficult or impossible to reverse.8

Compositional harm. As AgentHazard demonstrated, individually benign actions can compose into harmful trajectories. Each step passes safety filters, but the overall trajectory is dangerous. Current safety mechanisms evaluate actions in isolation, not in context.9

Capability overhang. Agents may have capabilities their designers don’t know about. A model with access to a code execution tool can, in principle, write and run arbitrary programs. The space of possible behaviors is combinatorially large, and testing can only cover a fraction of it.

Extended autonomy. Agents that operate for hours or days—like those in CORAL—accumulate context, develop strategies, and make decisions that are increasingly distant from any human instruction. The longer an agent runs, the more it is guided by its own reasoning rather than the user’s intent.

These challenges are not fatal—agents are enormously useful and the benefits may outweigh the risks—but they require a fundamentally different safety approach than chatbot safety. Alignment for agents is not about refusing harmful queries; it’s about maintaining safe behavior across extended, autonomous, tool-using trajectories.

Where We Are

LLM agents are, as of 2026, in a phase of rapid capability expansion and cautious deployment. Frameworks like Claude Code, OpenAI Codex, Devin, and open-source projects like OpenHands are pushing the boundary of what agents can do. The AI evaluation benchmarks are evolving to measure agent capabilities—SWE-bench for coding, WebArena for web tasks, OSWorld for computer use.

The trajectory is clear: agents will become the dominant modality for LLM deployment. The question is whether we can build them safely. The alignment problem was already hard for chatbots. For agents, it may be the defining challenge of the next decade.

Further Reading