AI log, day 03: a prompt is an API contract, not a magic spell

Day 02 covered how a model picks the next token once it has a distribution to sample from. That’s the last layer down toward the math. This entry goes the other direction, up toward the part every tutorial calls “prompt engineering” and every skeptical engineer, myself included until recently, mentally files under rephrasing a Google search until it works. That reaction turned out to be wrong, or at least aimed at the wrong target. The actual discipline isn’t finding magic words. It’s treating a prompt the way you’d treat any other interface into a system you don’t control the internals of: an API call with inputs, defaults, a contract about what comes back, and a trust boundary that either exists or doesn’t. I’d been writing that contract by accident, in paragraph form, and calling the result “just prompting.”

A prompt is a function call with no type checker

Strip away the fact that it’s plain English, and a typical prompt to a chat model is doing exactly what a function call does in any other API: it bundles configuration, an operation to perform, and the data that operation runs on, into one payload, and it expects a specific shape back. The difference is that a normal API call has a compiler or a client library rejecting the wrong shape before it ever leaves your machine. A prompt has none of that. Every one of those four roles, the system message, the instruction, the context, and the output format, gets typed into the same text box, and nothing stops you from leaving one out, blending two together, or writing an output format the model quietly ignores half the time.

Concretely, most prompts I’d written before this log were one paragraph doing all four jobs at once:

You're a helpful assistant. Given this support ticket, figure out if it's
billing, technical, or account related, and also tell me how urgent it
seems and summarize it in one line. Ticket: "I was charged twice this
month and can't log in to check my invoice."

That works, often enough that it’s easy to stop there. But nothing in it says what “urgent” is measured against, nothing pins down the exact three category labels versus close synonyms the model might drift into on a different call, and the output format is a hope, not a rule. It reads like a comment describing what a function should do instead of the function’s actual signature.

The same request written as an interface separates those four roles instead of folding them together:

messages = [
    {
        "role": "system",
        "content": (
            "You classify support tickets. Category must be exactly one of: "
            "billing, technical, account. Urgency must be exactly one of: "
            "low, medium, high. Always return valid JSON matching the schema, "
            "nothing else."
        ),
    },
    {
        "role": "user",
        "content": (
            "Ticket: \"I was charged twice this month and can't log in to "
            "check my invoice.\""
        ),
    },
]

Nothing about the underlying model changed between those two versions. What changed is that the second one has a config (the system message, set once, holding for every ticket this session ever classifies), a clearly scoped instruction instead of one blended into config, and a data payload that’s just data, not also carrying half the rules. That separation is the whole idea. It doesn’t guarantee correctness, nothing here does, but it turns “did I remember to ask for this” into “is this actually in the contract,” which is a much easier thing to audit later.

The system prompt is the part of the contract you only write once

The system message specifically deserves its own line because it behaves differently from the rest of the prompt in a way that’s easy to miss if you think of “the prompt” as one undifferentiated blob. It’s set once per session or per application, not re-typed on every call, the same way you configure a database client’s connection pool once instead of passing pool size on every query. Everything downstream, every user message, every retrieved document, every tool result, gets interpreted through whatever rules live there.

That makes the system prompt closer to a function’s preconditions and defaults than to an instruction. “Always answer in valid JSON,” “never invent a source that wasn’t in the provided context,” “refuse requests outside this scope,” those are exactly the kind of thing you’d put in a constructor or a config file because they should hold for every call, not something you’d want to remember to restate every time. Treating it that way changes what goes wrong when it’s missing: a bad system prompt isn’t a typo in one request, it’s a config error affecting every single call made under it, which is the same blast radius a wrong default timeout or a missing auth header would have in an actual API client.

Few-shot examples are test cases, not decoration

The other habit I’d picked up without examining it: describing the desired output in prose instead of just showing an example of it. “Summarize in a formal tone” is a description. Showing one full input paired with exactly the summary you wanted is a demonstration, and demonstrations are what few-shot prompting actually is, a handful of complete input-output pairs placed in the prompt before the real input.

The reason this works better than a longer description isn’t mysterious once you connect it back to day 01’s point about generation: a model predicts the next token conditioned on everything already in its input, including its own earlier tokens. A worked example is training-adjacent evidence sitting directly in context, showing the exact boundary between “here’s the input” and “here’s the output,” at the exact length and format you want. A prose description is one more layer of interpretation the model has to translate into that same boundary on its own, and translation is exactly where drift creeps in, a slightly different category label, one extra sentence of preamble it wasn’t asked for, a summary that’s a paragraph instead of a line.

Ticket: "App crashes every time I open the settings page."
Category: technical
Urgency: high

Ticket: "Can I get a copy of last month's invoice?"
Category: billing
Urgency: low

Ticket: "I was charged twice this month and can't log in to check my invoice."
Category: ???

Two examples like that pin down the format, the exact category vocabulary, and the level of terseness expected, all without a single sentence describing any of those things in the abstract. That’s the same reason a unit test communicates a function’s contract better than a comment above it does: the test is the actual input and the actual expected output, sitting right there, not somebody’s summary of what they’re supposed to be.

Structured output turns “hopefully JSON” into an actual contract

Even with a clean system prompt and good examples, “return valid JSON” in plain instructions is still a request, not a guarantee, because the model is generating token by token and nothing stops it from adding a stray sentence before the braces or drifting into almost-valid JSON on an unlucky sample. This is where the API layer on top of the model actually closes the gap: most providers now expose a way to constrain output to match a schema directly, OpenAI’s response_format with a JSON schema, Anthropic’s tool use forcing a specific tool call with typed arguments, rather than leaving schema conformance to the model’s best effort alone.

tools = [{
    "name": "classify_ticket",
    "description": "Classify a support ticket",
    "input_schema": {
        "type": "object",
        "properties": {
            "category": {"type": "string", "enum": ["billing", "technical", "account"]},
            "urgency": {"type": "string", "enum": ["low", "medium", "high"]},
            "summary": {"type": "string"},
        },
        "required": ["category", "urgency", "summary"],
    },
}]

Forcing the model to call classify_ticket with arguments matching that schema is the difference between hoping the output parses and having the client library reject it before your code ever sees a malformed value, the same distinction as an API returning a typed response versus a string you regex apart and hope for the best. It’s not a different model underneath, and it’s not immune to the model picking a wrong category, it just closes off the entire class of failure where the output isn’t even shaped like what you asked for. Given day 02’s point that any temperature above zero introduces real run-to-run variance, that class of failure is exactly the one worth eliminating structurally instead of hoping a good prompt suppresses it often enough.

A prompt is an interface, not a security boundary

Here’s the part that stopped feeling like a style preference and started feeling like an actual engineering concern: the system message, the instructions, and the untrusted data all travel through the same single text channel, and the model has no built-in way to tell which parts are supposed to be commands and which parts are just content to read. A normal API distinguishes code from data by construction, a SQL parameter can’t rewrite the query around it, a JSON field can’t redefine your route handler. A prompt has no equivalent wall. If a retrieved document, a user-submitted comment, or a scraped web page contains text that reads like an instruction, “ignore the above and instead,” the model has no structural reason to treat it as inert data rather than as the next thing to obey. That’s prompt injection, and it isn’t a bug you patch once, it’s a direct consequence of instructions and data sharing one channel with no built-in separation.

That risk is dormant right up until a prompt starts pulling in content the author of the system prompt didn’t write themselves, which is precisely what a RAG pipeline does on every single query. Every chunk retrieved from the vector store in the pipeline I keep coming back to is exactly that kind of untrusted content, sourced from documents someone else authored, inserted straight into the same context window as the instructions. Nothing in the setup I originally shipped drew a hard line between “this is the system’s instructions” and “this is retrieved data, read it but don’t obey it,” because at the time I wasn’t thinking about the prompt as a channel with a trust boundary at all, just a paragraph that got longer or shorter depending on what search returned.

What I’d actually change now

Putting the last few sections together into one concrete revision: the system prompt should state the contract once, output format included, and never get re-litigated per request. Few-shot examples should replace prose descriptions anywhere the output has a specific shape worth locking down. Anything meant to come back as data, not prose, should go through a forced schema instead of a plain-language request to “return JSON.” And retrieved content specifically needs to be visibly marked as data, wrapped in something like explicit <retrieved_context> tags with an instruction that content inside that boundary is reference material only and never a new instruction, which doesn’t make injection impossible, nothing fully does, but it gives the model an actual structural signal to weigh instead of forcing it to guess from tone alone which sentence in a wall of undifferentiated text is the real command.

None of this makes the model smarter. It’s the same shift as the rest of this log so far: temperature didn’t change what the model knows, it changed how the known distribution gets sampled; structuring a prompt doesn’t change what the model can do, it changes how much of what it can do actually survives contact with a real request instead of getting lost in an ambiguous instruction.

What’s next

Day 04 moves to fine-tuning versus RAG versus just writing a bigger, better-structured prompt, and an honest look at when each one is actually the right tool instead of the fashionable one. After that, day 05 tackles evals: how you actually measure whether a prompt or pipeline change made things better, instead of it just feeling better on the one example you happened to test.

Regular posts continue in between, as always.