You wire up a chatbot with a couple of function calls — one to hit a weather API, one to query a database — and you start calling it "an agent" in your Slack updates because that's what everyone else calls anything that touches a tool. Then someone asks it to "check the weather in three cities and email me if any of them will rain" and it falls over. It calls the weather API once, for one city, and stops. It doesn't retry, doesn't loop, doesn't know it has two more cities to check or that "email me" was even part of the task. It wasn't broken. It was never built to do that in the first place — because a function call bolted onto an LLM and an actual agent are not the same thing, even though the demo videos make them look identical.
What "Agentic AI" Actually Is
Think about the difference between a vending machine and a personal assistant running an errand. A vending machine takes one input and produces one fixed output — you press B4, you get the chips. There's no judgment involved, no revising the plan if B4 is out of stock. Now think about asking a personal assistant to "pick up dinner for six people, at least one vegetarian option, back by 7." That assistant checks a few restaurants, notices one is closed, adjusts, maybe calls ahead to confirm the vegetarian dish is actually available, and comes back when the job is done — not after one API-shaped action.
That second behavior is what "agentic" is pointing at. An agent isn't defined by having access to tools — plenty of non-agentic systems call APIs. What makes something an agent is that it runs a loop: it observes the current state of the world, decides what to do next based on that state, takes an action, observes the result of that action, and decides again — repeating until it judges the task done. The tool calls are just the hands. The loop is the thing doing the deciding.
This distinction matters because "agent" has become a marketing word slapped onto anything with function-calling enabled. A single LLM call that picks one tool and returns its output is a tool-using chatbot. It's useful, but it's a straight line, not a loop, and it can't recover from a step going wrong because there's no step after it to recover in.
The Core Technical Pieces
The observe-decide-act loop
This is the part that actually earns the word "agent." After every action, the result gets fed back into the model's context as new information, and the model decides the next action based on that updated state — not a plan it wrote once and executed blindly. If the weather API call fails, an agent notices the failure in the observation and can retry, try a different endpoint, or ask for clarification, instead of silently returning nothing.
Tool use / function calling
This is the vocabulary the loop speaks. The model is given a set of callable functions with typed schemas — a search tool, a code execution sandbox, a database query, a calendar write — and at each step it decides which one to call and with what arguments, expressed as structured output rather than free text. Tool use by itself is just the "act" step; without the loop around it, it's a single hand movement, not an errand.
Planning and task decomposition
Complex requests get broken into smaller steps, either explicitly (the model writes out a plan before executing) or implicitly (it decides the next single step at each iteration without ever writing the full plan down). Explicit planning is easier to inspect and debug — you can log the plan and see where it diverged from reality — but implicit, step-by-step decision-making tends to be more resilient to changing conditions, since the model isn't locked into a plan written before it knew a step would fail.
Memory
An agent needs to track what it's already tried and what it's learned mid-task, which is different from the LLM's context window in the same way a to-do list is different from short-term memory — it's explicit state, often built by the orchestrating code around the model rather than by the model itself. Longer-running agents also need memory that survives past one loop, so a multi-day task doesn't restart from zero every time it wakes up.
The control flow / orchestrator
Something outside the model has to run the loop: call the model, execute whatever tool it picked, feed the result back in, and decide when to stop — on success, on a hard iteration cap, or on a repeated failure pattern. This orchestrating layer is unglamorous but it's where most of the actual engineering lives, because it's what turns "an LLM that can call functions" into a system that runs unattended for more than one step.
A Real-World Example
Picture a customer support agent handling a refund request. Step one: it reads the ticket and looks up the order in the order-management system — that's an observation. Step two: it decides the order qualifies for a refund under policy, based on rules in its context, and calls the refund-processing tool — that's an action. Step three: the refund tool returns an error because the payment method on file has expired — a new observation. A non-agentic script would just fail here and hand the ticket to a human. The agent instead decides, based on that error, to look up the customer's updated payment method, or to draft a message asking them to confirm one, then tries the refund again once it has what it needs. Only once the refund actually succeeds — confirmed by a final observation — does it close the loop and reply to the customer.
Nothing in that flow required a genius model. What made it work was the loop giving the system a chance to see its own failure and adjust, instead of a single shot that either works or doesn't.
Why Not Just a Tool-Using Chatbot, or a Fixed Workflow?
A single tool-augmented LLM call is cheaper and far more predictable — one request, one response, easy to test, easy to put a strict timeout around. It's the right choice when the task really is one step: "look up this order's status" doesn't need a loop. The moment the task can branch or fail partway through, though, a single call has nowhere to go when things don't go as expected.
A fully scripted workflow — a decision tree or state machine written by a human — is the other alternative, and it's often the better one. It's deterministic, cheap to run, and trivial to unit test, because every path through it is known in advance. The tradeoff is that it can only handle the paths someone thought to code. An agent trades that predictability for flexibility: it can handle situations nobody explicitly programmed for, at the cost of being non-deterministic, harder to test exhaustively, slower (multiple model calls per task instead of one), and more expensive to run. The honest framing is that agents are worth the cost specifically when the space of possible situations is too large or too unpredictable to hand-code — not as a default upgrade over a workflow that already works.
A Pitfall Worth Knowing
The failure mode that catches people off guard isn't the agent being "dumb" — it's the agent not knowing when to stop. Without an explicit, well-defined success condition, an agentic loop can keep going: retrying a failing tool call with slightly different arguments, convincing itself progress is being made when it isn't, or oscillating between two actions that undo each other. Each iteration is another model call, so a runaway loop isn't just a logic bug — it's a bill, and in the worst case it's a bug with side effects, because a loop that keeps deciding to "try again" on a payment or a file-deletion tool doesn't just cost tokens.
The fix isn't a smarter model, it's guardrails around the loop itself: a hard cap on iterations, an explicit and checkable definition of "done" rather than relying on the model to self-report success, and a human-in-the-loop confirmation step before any action that isn't easily reversible. The agent pattern gives you flexibility; it's the orchestrator's job to make sure that flexibility has a floor.
The Takeaway
An agent is defined by the loop, not the tool call — perceive, decide, act, observe, repeat — with planning, memory, and an orchestrator holding the whole thing together and knowing when to call it done. That's also exactly where agents get expensive, unpredictable, and occasionally stuck, which is why "just make it an agent" is a much bigger decision than it sounds like in a product meeting.
Once you've got one loop working reliably, the next question is what happens when you need more than one — a planner agent handing off to a researcher agent, handing off to a writer agent, each with its own tools and its own loop. That's multi-agent orchestration, and it's next in this series.
