Solutions sit behind the flaps. Read the pattern, think, then open one step. If you already tried the problem, open everything.

The GSTIN Field That Was Sometimes A Sentence

A merchant onboarding team at a payments startup in Pune built a flow where a shopkeeper uploads a photo of their GST certificate, an OCR step reads the text, and a model extracts business name, GSTIN, PAN and bank account number into a form. Ninety five percent of the time it returned clean JSON. The team shipped it and moved to the next feature.

The other five percent looked like this.

{
  "businessName": "Sharma General Store",
  "gstin": "The GSTIN is 27AAAPL1234C1Z5, which follows the standard format.",
  "pan": "AAAPL1234C",
  "bankAccount": "123456789012"
}

Valid JSON. Every key present. The gstin field held a full sentence instead of a bare value. Their downstream code ran a regex check on that field expecting exactly 15 characters, and it failed silently, dropping the merchant into a manual review queue with no explanation. Over three weeks, around 400 genuine merchants sat stuck in that queue because a field that was supposed to hold data sometimes held a paragraph.

Nobody had written code to check for this, because nobody had asked the question that matters here. Getting valid JSON back is not the same as getting the JSON you designed for.

Structured output (getting a model's response to conform to a schema you define, instead of free running prose) is the fix, and it is deeper than adding the words "respond in JSON" to your prompt. By the end of this lesson you will know why that phrase alone does not work, what actually enforces the schema, and how to design a client interface for it that survives a change of model provider.

If you came here from the earlier foundation lessons, you already know that retries burn tokens, that partial wrong fields can look schema-valid, and that prompts are contracts in git. This lesson is where those ideas become a ModelClient you can implement in Java, .NET, Node, or Python.

Why Should You Care?

Every real integration needs code to read the answer, not a person. A chatbot that produces nice prose is a demo. A system that extracts a GSTIN, decides a refund amount, or picks which tool to call needs output your code can parse without guessing. This is where an AI feature becomes a system instead of a toy.
It is the single most asked practical question in AI engineering interviews right now. Not "what is a transformer". The question is closer to "the model sometimes returns malformed JSON in production, walk me through your fix." A weak answer says "add a retry." A strong answer explains schema enforcement, validation, and a bounded retry with a fail closed path.
Everything in Phase 3 depends on this working. An agent is a loop that reads a structured decision from the model and acts on it. If you cannot trust the shape of that decision, you cannot safely build an agent on top of it. This lesson is the floor the rest of the path stands on.

🟢 The Simple Version (Start Here If You're New)

A Form, Not A Letter

If you ask someone to describe their address in a letter, you get a paragraph. Readable, but your computer cannot pull the pin code out of it reliably, because the person is free to phrase it however they like.

If you hand them a form with labelled boxes, city, state, pin code, you get data. The same information, but now it is something a machine can process without guessing.

In plain words: asking a model for structured output means giving it the form instead of asking for the letter. You describe the exact shape you want, field names and types, and the model fills in that shape rather than writing you a paragraph.

Why This Is Different From "Please Reply In JSON"

Typing "respond only in valid JSON" into your prompt feels like the same thing. It is not, and the difference is the whole lesson.

A plain instruction is a request the model tries to honour, the same way it tries to honour every other instruction in your prompt. It can still add a sentence before the JSON. It can still wrap the JSON in a markdown code fence. It can still put a full sentence inside a field that was supposed to hold a 15 character code, exactly like the GSTIN example above.

In plain words: structured output done properly is not a polite request. Most providers today offer a mode where you hand over an actual schema, and the model's response is constrained to match it at generation time, not checked afterwards. The model is not being asked to behave. It is being limited to only the shapes that are valid.

Tool Calling Is The Same Idea, Aimed At Actions

Scroll sideways to read, or tap to zoom

Tool calling (also called function calling) is structured output where the shape describes an action instead of a data record. You describe a tool, for example blockCard(cardLastFour: string), and instead of writing prose, the model can respond with a structured request to call that tool with specific arguments.

Here is the part beginners get wrong. The model never calls the tool. It produces a structured payload that says it wants to. Your code reads that payload and decides whether to actually run blockCard. The model proposes. Your code disposes. That boundary is what keeps a support bot from actually blocking a customer's card because it misread a sentence.

Both ideas, structured data and tool calls, are solved by the same mechanism, and that mechanism is what the rest of this lesson explains.

🟡 Going Deeper

The Three Ways Teams Try To Get JSON, And Why Only One Works Reliably

Scroll sideways to read, or tap to zoom

Prompting alone. "Respond only with JSON matching this shape." Cheapest to try, least reliable. The model can still add commentary, wrap output in code fences, or drift on an edge case it was not trained to handle. This is what the Pune team was doing when the GSTIN field turned into a sentence.

Parse and retry. Ask normally, try to parse the response, and if parsing fails, ask again with the error included. Better than nothing, but every retry is a paid model call, and a badly designed schema can make every attempt fail the same way, which we come back to in the war story below.

Constrained decoding through a schema or tool definition. You give the provider an actual JSON schema. The provider restricts token generation so that only valid completions of that schema are possible. This is the real fix, and it is what "structured output" and "tool calling" mean in a modern provider API.

The third approach does not eliminate every failure mode. A field can still be schema valid and semantically wrong, which is the next section. But it eliminates the entire class of failure the GSTIN bug fell into, because the model is no longer free to put a sentence where a string field was declared.

Why Schema Valid Is Not The Same As Correct

This is the trap that catches teams who think structured output solved everything.

{
  "gstin": "27AAAAA0000A1Z5",
  "pan": "AAAAA0000A",
  "bankAccount": "000000000000"
}

Every field here is the right type, the right length, the right format. It could still be entirely invented. Schema constraints check shape. They say nothing about whether the GSTIN actually belongs to this merchant, or exists at all.

Shape validation happens for free once you use a schema. Correctness needs a second, separate check: a GSTIN checksum, a lookup against a government registry, a human review for anything that touches money. This is the same "model drafts, code verifies" pattern from Lesson 1, applied specifically to the fields inside a structured response rather than to the response as a whole.

The Interface Worth Building Once

Scroll sideways to read, or tap to zoom

Every provider names this feature differently, and the exact method calls will keep changing. What should not change is the shape of your own abstraction. Define one interface, in whichever language you use, and build every provider integration behind it.

interface ModelClient {
  // Ask the model to produce a value matching the given schema.
  // Implementations should prefer the provider's native structured output
  // or tool calling mode. If that mode is unavailable, fall back to prompting
  // for JSON and validating strictly on the way out, never trust unchecked.
  generateStructured<T>(
    prompt: string,
    schema: JsonSchema,
    options?: { maxRetries?: number }
  ): StructuredResult<T>
}

// A result type that forces the caller to handle failure, instead of a
// value that might silently be null or half filled.
type StructuredResult<T> =
  | { ok: true, value: T }
  | { ok: false, error: string, rawOutput: string, attempts: number }

Two things this interface buys you. First, when the provider changes its API, or you add a second provider for fallback, only the implementation behind ModelClient changes, nothing that calls it. Second, StructuredResult forces every caller to handle the failure branch, instead of quietly assuming value is always populated, which is exactly the assumption that let the GSTIN bug reach production.

Below is the same interface satisfied in four ecosystems. The pattern is identical everywhere: define the schema, call the model, get back a typed value or a typed failure. Framework method names will drift over time. The shape of the contract should not.

Java (Spring AI)

// Illustrative: exact method names vary by Spring AI version, the
// interaction pattern (schema in, typed record or error out) does not.
public sealed interface StructuredResult<T> {
    record Ok<T>(T value) implements StructuredResult<T> {}
    record Failed<T>(String error, String rawOutput, int attempts) implements StructuredResult<T> {}
}

public record MerchantDetails(
    String businessName,
    String gstin,
    String pan,
    String bankAccount
) {}

public class SpringAiModelClient implements ModelClient {

    private final ChatClient chatClient;

    public <T> StructuredResult<T> generateStructured(
            String prompt, Class<T> targetType, int maxRetries) {
        for (int attempt = 1; attempt <= maxRetries; attempt++) {
            try {
                // .entity() asks Spring AI to constrain the response to the
                // shape of targetType using the provider's structured output mode.
                T value = chatClient.prompt(prompt)
                    .call()
                    .entity(targetType);
                return new StructuredResult.Ok<>(value);
            } catch (StructuredOutputException e) {
                if (attempt == maxRetries) {
                    return new StructuredResult.Failed<>(
                        e.getMessage(), e.getRawResponse(), attempt);
                }
                // retry with the failure appended, never loop unbounded
            }
        }
        throw new IllegalStateException("unreachable");
    }
}

.NET (Microsoft.Extensions.AI)

// Illustrative: Microsoft.Extensions.AI and Semantic Kernel both expose a
// JSON schema response format. Exact type names move between previews.
public record MerchantDetails(
    string BusinessName,
    string Gstin,
    string Pan,
    string BankAccount
);

public abstract record StructuredResult<T>
{
    public sealed record Ok(T Value) : StructuredResult<T>;
    public sealed record Failed(string Error, string RawOutput, int Attempts) : StructuredResult<T>;
}

public class DotnetModelClient : IModelClient
{
    private readonly IChatClient _chatClient;

    public async Task<StructuredResult<T>> GenerateStructuredAsync<T>(
        string prompt, int maxRetries = 2)
    {
        for (int attempt = 1; attempt <= maxRetries; attempt++)
        {
            var options = new ChatOptions
            {
                // Constrains generation to the JSON schema of T, enforced
                // by the provider, not just requested in the prompt text.
                ResponseFormat = ChatResponseFormat.ForJsonSchema(
                    JsonSchemaBuilder.Build(typeof(T)))
            };

            var response = await _chatClient.GetResponseAsync(prompt, options);

            if (TryParse<T>(response.Text, out var value, out var error))
                return new StructuredResult<T>.Ok(value!);

            if (attempt == maxRetries)
                return new StructuredResult<T>.Failed(error!, response.Text, attempt);
        }
        throw new InvalidOperationException("unreachable");
    }
}

Node / TypeScript (Vercel AI SDK)

// Illustrative: generateObject already returns this contract shape almost
// exactly. This wraps it so callers only ever see StructuredResult.
import { z } from 'zod';
import { generateObject, NoObjectGeneratedError } from 'ai';

const MerchantDetails = z.object({
  businessName: z.string(),
  gstin: z.string().length(15),
  pan: z.string().length(10),
  bankAccount: z.string(),
});

type StructuredResult<T> =
  | { ok: true; value: T }
  | { ok: false; error: string; rawOutput: string; attempts: number };

async function generateStructured<T>(
  prompt: string,
  schema: z.ZodType<T>,
  maxRetries = 2
): Promise<StructuredResult<T>> {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      // schema is enforced by the provider's structured output mode, not
      // just described in the prompt text.
      const { object } = await generateObject({ model, prompt, schema });
      return { ok: true, value: object };
    } catch (err) {
      if (attempt === maxRetries || !NoObjectGeneratedError.isInstance(err)) {
        return {
          ok: false,
          error: String(err),
          rawOutput: NoObjectGeneratedError.isInstance(err) ? err.text : '',
          attempts: attempt,
        };
      }
      // fall through, retry with the same schema
    }
  }
  throw new Error('unreachable');
}

Python (LangChain)

# Illustrative: with_structured_output wraps a provider's native structured
# mode when the model supports it, and falls back to prompted JSON parsing
# with validation when it does not. Know which mode you are actually in.
from pydantic import BaseModel
from dataclasses import dataclass
from typing import Generic, TypeVar

T = TypeVar("T")

class MerchantDetails(BaseModel):
    business_name: str
    gstin: str
    pan: str
    bank_account: str

@dataclass
class Ok(Generic[T]):
    value: T

@dataclass
class Failed:
    error: str
    raw_output: str
    attempts: int

def generate_structured(prompt: str, schema: type[BaseModel], max_retries: int = 2):
    structured_model = model.with_structured_output(schema, strict=True)
    last_raw = ""
    for attempt in range(1, max_retries + 1):
        try:
            value = structured_model.invoke(prompt)
            return Ok(value)
        except Exception as exc:
            last_raw = getattr(exc, "raw_output", "")
            if attempt == max_retries:
                return Failed(str(exc), last_raw, attempt)
            # retry, do not loop unbounded
    raise RuntimeError("unreachable")

Notice what stays constant across all four. A schema goes in. A typed value or a typed failure comes out. Nothing in any of these four blocks is interesting on its own, and that is the point. The interesting decision, the ModelClient contract, was made once, in the neutral block above, and every language just satisfies it.

🔴 Architect's Corner

The Retry Loop That Cost Real Money

I have seen a team ship a schema with a typo in a regex pattern for a PIN code field, six digits required, and the pattern they wrote could never match six digits due to an off by one in the character class. Every single call against that schema failed validation. Their retry logic, written before anyone thought carefully about bounds, retried five times on every failure with no circuit breaker.

For three days, every request to that endpoint made five model calls instead of one, and every one of the five failed the same way, because the schema itself was broken, not the model's output. Nobody noticed because the retries succeeded eventually, in the sense that the request did not hang, it simply returned a final failure to the user after paying for five calls first.

The fix was two things. First, validate your schema against real data in a test before you ship it, the same discipline you would apply to a database migration. Second, distinguish a schema bug, which retrying can never fix, from a model drift, which retrying sometimes can. If the exact same validation error repeats on attempt one and attempt two with unchanged input, stop retrying immediately and alert, rather than burning the full retry budget every time.

Flat Schemas Beat Nested Ones

The deeper a schema nests, the more places a model can drift. A flat schema with ten string and number fields is close to reliable across most providers today. A schema with three levels of nested objects and arrays of objects inside arrays degrades noticeably, especially on smaller or cheaper models.

If accuracy on a complex extraction matters more than call count, split it. Extract the top level fields in one call, then make a second call per nested section rather than one call trying to hold the whole document's structure at once. This costs more calls and buys you meaningfully higher reliability on each one. Whether that tradeoff is worth it depends entirely on what a wrong field costs you downstream.

Offering Too Many Tools At Once

The same reliability curve shows up in tool calling. Offer a model three well described tools and it picks correctly almost every time. Offer it thirty, several with overlapping purposes, and tool selection accuracy drops, because the model now has to disambiguate between tools that look similar from the description alone.

Group tools by context and only expose the ones relevant to where the conversation currently is, rather than handing over your entire tool catalog on every call. This is also a security boundary, not just an accuracy one, and gets full treatment in Lesson 18 when we cover tool authorization.

Structured Modes Are Not Always The Same Latency Or Cost

Constrained decoding is not free. Depending on the provider, enforcing a schema can add latency compared to unconstrained generation, because the decoding process has extra bookkeeping at each token step. It is very rarely the wrong tradeoff for anything that feeds a downstream system, but budget for it rather than assuming structured calls cost the same as free text ones. We build the full latency budget in Phase 5.

How India-scale systems do it

Merchant onboarding at Indian fintech and payments companies is exactly the scenario this lesson opened with, and it is a genuinely hard structured extraction problem. A GST certificate photographed on a shopkeeper's phone, sometimes in poor light, sometimes with regional language text mixed with English, produces OCR output that is noisy before a model ever sees it. Schemas for this kind of extraction have to make most fields optional rather than required, because forcing every field to be present on a low quality photo just produces more invented data to fill the gap, not more real data.

WhatsApp support bots that extract intent and then call a real tool, block a card, raise a refund, escalate to a human, are the clearest case for why the model proposes and code disposes boundary matters. A Hinglish message like "mera card block kar do, kisi aur ne use kiya hai" has to become a structured call to blockCard with the right card reference, not a free text reply, and your code has to confirm the card actually belongs to the person messaging before it runs that call.

Cost compounds fast at Indian volume. A merchant onboarding pipeline processing lakhs of signups a month, each with a two or three attempt retry budget on a schema that occasionally fails, turns into a meaningful line item if nobody is watching the retry rate. The fix from the war story above, catching a broken schema before it burns budget on every call, matters more at this scale, not less.

Tradeoff: stricter schemas and lower retry budgets produce fewer bad records reaching your database, and a higher fraction of genuinely valid extractions get rejected into a manual review queue instead of being accepted, which needs enough human reviewers to keep up.

The Decision Matrix

Your situation	Approach	Reasoning	Indian example
Extracting a few flat fields, provider supports structured output	Native structured output mode	Most reliable, enforced at generation, not checked after	Merchant name and GSTIN from a chat message
Provider or model does not support structured modes	Prompt for JSON plus strict validation and bounded retry	Best available fallback, must fail closed	Legacy or self hosted models without constrained decoding
Deeply nested document, accuracy critical	Split into multiple flat calls	Nested schemas degrade reliability, flat calls do not	Full KYC document with multiple sections
Model should perform a real action	Tool calling, with your code executing the call	Keeps the model proposing and your code deciding, not the model acting directly	WhatsApp bot blocking a card on request
Field feeds a financial or compliance decision	Schema validation plus a separate semantic check	Schema valid does not mean correct, checksum or lookup catches invented values	GSTIN checksum, PAN format check before storing
High volume, cost sensitive	Cap retries hard, alert on repeated identical failures	A broken schema fails every attempt the same way, retrying it is pure waste	Lakhs of merchant signups a month

Common Mistakes

1. "Tell it to reply in JSON and we're done." This is the GSTIN bug from the opening story. A plain instruction is a request, not a constraint. Use the provider's actual structured output or tool calling mode wherever it exists.

2. "It passed schema validation, so the data is correct." Schema validation checks shape. It says nothing about whether a GSTIN is real, whether an amount matches a policy, or whether a name is invented. Add a second, separate correctness check for anything that matters.

3. "Retry until it works." A retry loop with no cap and no circuit breaker on repeated identical failures will burn your budget on a broken schema for as long as nobody notices, exactly as it did in the war story above.

4. "One big schema captures the whole document." Nested schemas degrade reliability faster than flat ones. Split extraction into multiple calls when accuracy on a complex document actually matters.

5. "Tool calling means the model runs code." It does not. The model proposes a call. Your code decides whether to execute it, with whatever authorization and validation that action deserves. Treating a tool call payload as already authorised is how a misread message turns into a real world action.

🧠 Key Takeaways

Structured output is enforced at generation, not requested in prose. "Reply in JSON" is a request the model can still ignore in small ways. A real schema, using the provider's structured output or tool calling mode, constrains what tokens are even possible.
Schema valid is not the same as correct. Shape checking is free once you use a schema. Correctness needs its own check, applied after.
Cap your retries, and watch for identical repeated failures. That pattern means your schema is broken, not the model, and retrying will never fix it.
The model proposes a tool call, your code decides to run it. That boundary is a security control, not a technicality, and it gets deeper treatment in Lesson 18.
One interface, four implementations. ModelClient stays stable while every provider's SDK changes underneath it. Build the abstraction once.

Think About It

Your team's schema for extracting a delivery address has 12 required fields. Real addresses from tier two Indian cities routinely omit two or three of them, apartment number, landmark, pin code. What would you change about the schema itself, before touching any retry logic?
A colleague says "we don't need tool calling, we'll just have the model write the SQL query directly and run it." What is the strongest argument against that, using the model proposes, code disposes idea from this lesson?
Your retry rate on a structured extraction endpoint jumps from 2 percent to 40 percent overnight, with no code deploy on your side. Walk through how you would tell whether the cause is a model change (Lesson 1 revisited) or a broken schema (this lesson's war story), and what you would check first.

Structured Outputs and Tool Calling

The GSTIN Field That Was Sometimes A Sentence

Why Should You Care?

🟢 The Simple Version (Start Here If You're New)

A Form, Not A Letter

Why This Is Different From "Please Reply In JSON"

Tool Calling Is The Same Idea, Aimed At Actions

🟡 Going Deeper

The Three Ways Teams Try To Get JSON, And Why Only One Works Reliably

Why Schema Valid Is Not The Same As Correct

The Interface Worth Building Once

🔴 Architect's Corner

The Retry Loop That Cost Real Money

Flat Schemas Beat Nested Ones

Offering Too Many Tools At Once

Structured Modes Are Not Always The Same Latency Or Cost

How India-scale systems do it

The Decision Matrix

Common Mistakes

🧠 Key Takeaways

Think About It

Further Reading

Learn it in your stack