What an Agent Actually Is: Loop, Tools, Memory, Stop

The Support Bot That Kept Booking Tickets

A fintech support bot in Gurgaon was supposed to help customers with card blocks, UPI disputes, and statement downloads. Product called it an "agent" on the slide deck. Engineering had wired one model call, three tools, and a generous max-token budget. Demo day looked smooth.

Two weeks into production, a customer messaged on WhatsApp: "I need last three months of statements and also check if any chargebacks are pending." The bot fetched statements. Then it called the chargeback tool. Then it fetched statements again. Then it tried to open a dispute for a transaction the customer never mentioned. Then it fetched statements a third time. Eight model rounds later the session hit the provider rate limit, the customer got a vague apology, and finance later found the account had three duplicate dispute tickets.

Nobody had written a real stop condition (a rule that says "we are done, stop calling the model"). The loop ran until something external broke. Cost for that one chat sat near four rupees. Multiply that by a bad day of confused multi-intent chats and you are not running support, you are running a small bonfire of tokens.

I have seen this pattern more times than I want to admit. The word "agent" shows up in the pitch, and the code becomes "let the model decide what to do next" with almost no product logic around it. That is not architecture. That is hope with an API key.

By the end of this lesson you will be able to draw the agent loop on a whiteboard, name the four parts that actually matter (loop, tools, memory, stop), and argue when you should not build an agent at all.

Why Should You Care?

1. Interviews ask for the difference between a chatbot, a workflow, and an agent. A weak answer waves at "autonomy." A strong answer talks about a controlled loop, tool proposals, and hard stop rules.
2. This is a real architecture fork. Open-ended agents can handle messy tasks, but they trade predictability and cost control. Most production systems in Indian product companies should start as fixed workflows with a model step inside, not free-running agents.
3. The production pain is quiet and expensive. Loops without caps, tools without ownership, and memory that grows forever show up as bill spikes and weird customer outcomes, not as clean 500 errors.

🟢 The Simple Version (Start Here If You're New)

The Delivery Boy With A List And A Phone

Think of a delivery boy who has a list of stops, a phone to call the warehouse, and a rule that says "come back when the list is empty or when the shop closes." He does not invent new cities to visit just because he still has battery left.

In plain words: an agent (in AI systems) is a loop that asks a model what to do next, may run tools (actions your code can take, like fetch balance or raise a ticket), keeps some memory of the run, and must hit a stop condition so it does not run forever.

while not done:
  model sees: goal + memory + tool results so far
  model replies: final answer OR "call tool X with args Y"
  if final answer: stop
  if tool call: your code may run it, then append the result to memory
  if step count or budget exceeded: stop with a safe failure

That is the whole skeleton. Everything else is product rules wrapped around it.

Chatbot, Workflow, Agent

Three things people mix up in the same meeting.

In plain words: a chatbot is mostly "user message in, model text out." One shot, maybe a little history. No tools required.

In plain words: a workflow (or pipeline) is fixed steps your code owns. Step 1 classify intent, step 2 retrieve policy, step 3 call model for a reply, step 4 validate. The graph is yours. The model is a worker inside a box.

In plain words: an agent lets the model choose the next step, often which tool to call, from a set you expose. Control is shared. The model proposes. Your code still executes and still enforces limits.

Most "agents" I see in demos should have been workflows. A UPI dispute flow with five known steps does not need open-ended planning. It needs reliable stages, clear tools, and a human handoff. Save the open loop for tasks where the path truly cannot be listed up front.

The Four Pieces You Must Name

1. The loop. Each turn is a model call. More turns mean more latency and more rupees.

2. Tools. The model does not magically talk to your database. It proposes a structured tool call. Your code decides whether to run it. You already met this idea in structured outputs and tool calling. Agents just do it repeatedly.

3. Memory. Whatever you put back into the next call: conversation so far, tool results, a short plan. Memory is not free. Every remembered token is paid again on the next round.

4. Stop. Max steps, max spend, success criteria, user cancel, or "I do not know, escalate." Without stop, you do not have a system. You have a process that ends when the cloud provider gets angry.

What "Done" Means

The model saying "I am done" is not enough on its own. Your stop rules should be code-side:

• Max tool rounds (for example 5 for support, 15 for a research job)
• Max total tokens or max rupees for this run
• Required success signal (ticket id created, balance returned, user said thanks)
• Hard fail into a human queue when the model loops on the same tool with the same args

That is enough to use the word "agent" without lying to yourself. Next we walk the loop with real numbers and the places teams get burned.

🟡 Going Deeper

: The Loop With Real Costs

One Trace, One Bad Day

Customer: "Block my card ending 4521 and send me last month's statement on email."

A disciplined agent run might look like this:

Step 1  model -> tool: listCards(customerId)
        code  -> returns two cards, ****4521 and ****8890
Step 2  model -> tool: blockCard(cardId=...4521, reason=user_request)
        code  -> checks ownership, blocks, returns ok
Step 3  model -> tool: emailStatement(month=last, channel=email_on_file)
        code  -> queues email, returns job id
Step 4  model -> final: "Card ending 4521 is blocked. Statement email is queued."
Stop    success after 4 model calls

Cost sketch at Indian support volumes (illustrative, not a vendor quote):

Now the same request with a sloppy loop: no dedupe of tool results, no max steps, model re-lists cards twice "to be sure." You can double cost without improving the outcome. At 50,000 chats a day, that is not a rounding error.

Workflow First, Agent Second

I push teams to write the happy path as a workflow first.

classify intent
  -> if block_card: verify identity -> list cards -> confirm -> block -> notify
  -> if statement: pick period -> generate -> deliver
  -> if unknown: retrieve FAQ or escalate

Then ask: where does the path truly branch in ways we cannot list? That is where a small agent loop earns its keep. "Customer sent a messy Hinglish paragraph mixing three intents and a complaint about an old ticket" might need a planner. "Customer tapped Block Card in the app" does not.

Memory In One Screen

Short term memory for an agent run is usually:

• The user goal for this session
• Prior assistant messages you choose to keep
• Tool results, ideally summarised
• A scratch note like "already blocked 4521"

Long term memory (profile facts, past tickets) is a separate design. We cover it in the next lessons. For this lesson, remember one line: if you dump every raw tool payload into the next prompt, your cost grows roughly with the square of your messiness, because each step re-reads everything you already paid for.

Stop Conditions That Survive Production

Soft stop: the model returns a final message with no tool call.

Hard stops your code must own:

1. Step budget. if steps >= MAX_STEPS: escalate
2. Money budget. Track estimated tokens or a rupee ceiling per session.
3. Repeat detection. Same tool name + same args twice in a row with no new user input is a smell. Abort or force a different path.
4. Auth failure. Tool returned 403. Do not let the model invent another tool that "might work."
5. User interrupt. Customer says "ruk jao" or closes the chat.

I once reviewed a "research agent" that had max steps set to 50 because "research is hard." In practice it spent 40 steps reformatting the same PDF extract. The correct max for that product was 8 with a clear "partial answer + sources" exit.

Agent Equals Risk Surface

Every tool you expose is an API with a natural language front door. A workflow calls blockCard only from the block-card step. An agent can attempt blockCard on turn one because the user said something angry. That is why tool design and authorization get their own lesson next. The mental model starts here: the loop multiplies whatever power you hand the model.

A Minimal Loop You Can Implement Monday

Here is the skeleton I want juniors to type once by hand before they open LangGraph or Semantic Kernel. Language does not matter. The control points do.

function runAgent(goal, session):
  state = loadSession(session)          // scratchpad + short history
  for step in 1..MAX_STEPS:
    if estimateRupees(state) > MAX_RUPEES:
      return fail("budget", state)

    reply = model.complete(
      system = SYSTEM_WITH_TOOL_SCHEMAS,
      messages = projectMessages(state),
      tools = listToolsFor(session)     // already filtered
    )

    if reply.isFinalAnswer:
      return success(reply.text, state)

    if reply.isToolCall:
      if isRepeat(state, reply.toolCall):
        return fail("repeat_tool", state)

      result = toolExecutor.execute(session.principal, reply.toolCall)
      state.appendTool(reply.toolCall, result)
      saveSession(session, state)
      continue

    return fail("empty_or_invalid_model_output", state)

  return fail("max_steps", state)

Notice what is not in the model:

• MAX_STEPS and MAX_RUPEES
• listToolsFor filtering
• toolExecutor.execute authorization
• isRepeat detection

If a framework hides those, you still need the same checks around it. The framework is glue. These five lines are the product.

How To Brief Product On Agents

When a PM says "we want an agent," translate to questions they can answer without model jargon:

1. What is a successful end state in one sentence?
2. Which actions may happen without a human?
3. What is the max time and max money per session we accept?
4. What happens when the system is unsure?
5. Which intents are 80 percent of traffic and can be a boring workflow?

If they cannot answer those, you are not ready to pick tools or frameworks. You are still doing product design, which is fine, and cheaper than a runaway loop in production.

🔴 Architect's Corner

Most "Agents" Should Be Workflows

I will say this plainly. If you can draw the steps on a whiteboard in under five minutes, ship a workflow. Use the model for the steps that need language: classification, extraction, drafting a reply. Keep routing in code.

Agents win when:

• The task is open-ended and the path varies a lot (ops runbooks with dozens of rare branches)
• Tool choice depends on intermediate results you cannot pre-graph cheaply
• A human is still in the loop for irreversible actions

Agents lose when:

• Actions move money, block cards, or change KYC state without a hard gate
• Latency must stay under a couple of seconds for WhatsApp-style chat
• Cost must stay well under a rupee per resolution and volume is high

The Hidden Product Decision

Product often wants "one agent that does everything." That forces a huge tool catalog, long prompts, and weak tool selection. Split products instead: collections agent, disputes agent, KYC agent, each with a small tool set and its own stop budget. Shared platform, separate loops.

Observability Or You Are Flying Blind

Log each step as a span: model call id, tools proposed, tools executed, tokens in and out, stop reason. When cost spikes, you need to answer "which tool loop burned us" in minutes. Without that, finance only sees a bigger OpenAI or Azure line item.

Autonomy Is A Slider, Not A Badge

Level 0: model drafts text, human sends. Level 1: model may call read-only tools. Level 2: model may call write tools after code-side policy checks. Level 3: model may call write tools that need human approval for high risk. Level 4: full auto on low risk paths only.

Pick a level per action class, not one level for the whole company. Blocking a card is not the same risk as fetching FAQs.

What Marketing Will Not Tell You

Framework demos show an agent booking flights in one take. Production India support has partial outages, flaky internal APIs, Hinglish, and customers who change their mind mid-flow. Your stop conditions and fallbacks are the product. The model is one component.

Partial Failure Inside The Loop

Real tools fail halfway. blockCard succeeds and emailStatement times out. A naive agent will retry the whole plan and block the card again if you lack idempotency. A better loop:

1. Scratchpad records which subgoals are done
2. On retry or next step, the model (or your workflow) sees "card already blocked"
3. Only the failed branch continues

This is ordinary distributed systems thinking. Agents do not exempt you from it. If anything, free-form loops make partial failure more common because step order is not fixed.

Latency Budgets For Multi-Step Runs

Single model call p95 might be 1.5 to 3 seconds with streaming. Three serial tool rounds with two model decisions between them can land at 8 to 15 seconds before the user sees a final answer. On WhatsApp that sometimes works. In an in-app checkout help drawer it does not.

Design options when the budget is tight:

• Collapse to a workflow with parallel tool fetches and one model draft at the end
• Speculatively fetch the two most likely tools before the model picks (waste some backend, save a round trip)
• Stream a status line ("checking your cards...") so perceived wait drops even if total time does not

Do not discover this on launch week. Measure a three-step trace in staging with production-like tool latency.

Cost Attribution Per Stop Reason

When you log stop reasons, graph cost by reason weekly:

If max_steps and repeat_tool dominate spend, you do not have a model quality problem first. You have a loop design problem. I have sat in reviews where leadership wanted a bigger model, and the table above saved us from that expensive wrong turn.

How India-scale systems do it

UPI and payments support. Intent is often a small closed set: transaction status, dispute, limit change, account link issues. Mature teams run classifiers and fixed playbooks. A free agent that can both reverse a transaction and open a marketing ticket is a compliance problem waiting to happen.

WhatsApp banking bots. Latency and cost matter. Many flows are button-driven workflows with a model only on free-text turns. When free text arrives, a short agent loop with two or three read-only tools is common. Write actions go through OTP or a human desk.

E-commerce during sale days. "Where is my order" is retrieval plus a status API, not an open planner. Returns and exchanges get more branching, still usually a state machine with model-assisted message drafting.

Internal ops agents. This is where open loops earn money: "check why settlements failed for merchant X yesterday." Engineers accept higher latency and cost if the agent has strong stop rules and cannot push to production systems without approval.

Tradeoff at India scale: open-ended agents raise the ceiling on messy tickets and raise the floor on cost variance. Fixed workflows keep unit cost predictable under a rupee when you design carefully, but they need product and eng to map the boring paths that make up most traffic.

The Decision Matrix

Common Mistakes

1. "If we call it an agent, the model will figure out the process." The model will figure out a process. It may not be your process. Encode the process you need in workflows and policies, then leave only residual freedom to the loop.

2. "Stop when the model says it is finished." Soft stop is fine as one signal. Hard budgets and repeat detection are mandatory. The Gurgaon bot "finished" only when the rate limiter finished it.

3. "More tools make a smarter agent." More tools make a confused agent and a larger attack surface. Prefer narrow tools and more of them over one god tool named doAnything.

4. "Keep full tool JSON in memory forever." You pay for it on every later step. Summarise results. Drop raw payloads you no longer need.

5. "Autonomy means no human in the loop." For high risk actions in fintech and health, autonomy without approval is not modern. It is negligent.

🧠 Key Takeaways

• An agent is a loop with tools, memory, and a stop condition, not a magic employee.
• Your code still executes tools and still enforces budgets. The model proposes next steps.
• Most production problems want workflows with model steps, not open-ended agents.
• Stop rules belong in code: max steps, max rupees, repeat detection, auth failures.
• Every extra step multiplies cost and latency. Design the happy path to finish in few rounds.
• Split big agent dreams into small specialist loops with small tool sets.

Think About It

1. Your PM wants one WhatsApp "super agent" for lending: KYC status, EMI questions, foreclosure quotes, and hardship requests. Which parts would you keep as workflows, and which would you allow into an open loop?

2. A run shows the same getTransaction tool called five times with the same id. What stop rule would you add, and what would you return to the customer?

3. Leadership asks for "full autonomy" on refunds under 500 rupees. What level on the autonomy slider do you actually ship on day one, and what metrics would make you raise it?

Further Reading

• Anthropic: Building effective agents: clear push toward simple patterns over elaborate multi-agent theatre
• OpenAI: Agents guide: practical loop and tool framing from a major provider
• Simon Willison on agents: short, skeptical notes that age better than hype posts

Learn it in your stack

• Java: Spring AI advisors and tool calling on ChatClient, or LangChain4j AiServices with tools. Build the while-loop yourself first so you see stop conditions in your code, not only in a framework.
• .NET: Microsoft.Extensions.AI function calling, or Semantic Kernel with a planner you tightly bound. Prefer explicit loops for support bots until you need graph features.
• Node / TypeScript: Vercel AI SDK generateText with tools and maxSteps, or LangChain.js agent executors. Cap maxSteps in config, not in a comment.
• Python: LangChain tool calling loops or LangGraph for explicit state machines. LangGraph shines when you want the workflow-and-agent mix this lesson recommends.