The Three Roles
System prompt. Set by the application developer or deployer, not the end user. It loads first, before any user input, and establishes the model's operating context: its persona, its constraints, what it should and shouldn't do, how it should format responses. In most enterprise deployments, the end user never sees it. It's the configuration layer.
User prompt. The message from the human in each conversational turn. What the user actually types. In a single-turn interaction, this is the entire input. In a multi-turn conversation, it's the most recent human message.
Assistant turn. The model's response. In a multi-turn conversation, prior assistant turns are included in the conversation object alongside prior user messages — the model receives the full history on every request, not just the most recent exchange.
These three role types compose the conversation object that gets sent to the model on each request. The model sees all of it, every time.
How It Works
Take a federal agency deploying an AI assistant for acquisition support. A contracting officer opens the interface and types: "What are the FAR requirements for sole-source justification?"
What actually gets sent to the model is not that question alone. It's a structured object that begins with a system prompt the contracting officer never sees — something that might run 800 to 1,500 tokens of instructions establishing that the model is an acquisition support assistant, that it should cite regulatory sources, that it should not provide legal advice, that it should escalate ambiguous situations to a human reviewer. Then comes the user's question. If this is turn three of a conversation, the two prior exchanges appear in the object as well, each tagged with its role.
The model processes the entire object and generates the next assistant turn. On the following request, the conversation object grows by one exchange. This is why context window capacity matters in practice — a long conversation with a large system prompt will eventually exhaust available context, and the application has to decide what to drop. That's covered elsewhere. The system prompt is not a one-time instruction; it's a persistent presence in every request for the life of the conversation.
A system prompt for a serious enterprise deployment isn't a sentence or two. It's closer to a specification: role definition, behavioral constraints, output formatting rules, examples of correct and incorrect responses, handling instructions for edge cases. The engineering work is in that document.
Okta Concept Mapping
The system prompt maps most naturally to policy configuration in an IAM system — specifically the kind of role-based policy that shapes what a user can do before they ever make a request. Both are set by administrators, not end users. Both are invisible to the user. Both define the operating envelope of a session. The analogy holds well enough to be useful.
Where it breaks: IAM policy is deterministic enforcement. If your role doesn't include write access to a resource, you don't get write access. Full stop. A system prompt is probabilistic guidance. You can write a system prompt that instructs the model never to discuss competitors, and the model will sometimes discuss competitors anyway — particularly under adversarial prompting or unusual phrasing. The model doesn't enforce instructions the way a policy engine enforces rules. It influences behavior. That distinction matters enormously in enterprise risk conversations, and it's the first thing a CAIO will probe.
When You'll Need This
When a federal IT director asks how the agency ensures its AI assistant stays on topic, doesn't hallucinate policy, and refuses to answer questions outside its authorized scope — that question has an answer, and the answer is system prompt engineering. Not model selection. Not fine-tuning. The behavioral constraints live in the prompt.
Consumer AI and enterprise AI look similar from the outside. The ChatGPT interface most people have used gives them a minimal or invisible system prompt and a blank user field. Enterprise AI features are built by writing the system prompt carefully, testing it against adversarial inputs, iterating on the instructions, and accepting that the result is a probabilistic improvement in consistency, not a guarantee.
That last point deserves some weight. Prompt engineering is a real discipline — there are practitioners who are meaningfully better at it than others, and the difference shows up in model behavior. But it operates on a system that is fundamentally stochastic. A well-engineered prompt dramatically narrows the distribution of outputs. It doesn't collapse it to a point.
When a buyer asks whether their AI assistant can be "locked down" to only answer acquisition questions, the honest answer is: system prompt engineering gets you most of the way there, and additional safeguards — output filtering, retrieval constraints, human review workflows — cover the remainder. The system prompt is the foundation. It's not the whole structure.
The gap between "I typed something into ChatGPT" and "we are building a prompt-driven enterprise feature" is exactly the width of a well-engineered system prompt. That's what your buyers are paying for when they buy an AI product, whether the vendor makes it explicit or not.