Two approaches dominate the conversation when enterprise buyers start asking how to make a large language model behave the way they want: prompting a frontier model with carefully constructed instructions, or fine-tuning a base model on domain-specific data. You'll encounter both in discovery calls, in RFIs, and in the specific meeting where a technical stakeholder says "we need to train our own model on our proprietary data." Knowing what each approach actually does — and what it structurally cannot do — is what separates the seller who earns the next meeting from the one who nods along and loses the deal to someone who asked a better question.
Frontier-Model Prompting
What it is: Directing a pre-trained, fully deployed model's behavior through structured natural-language input at inference time.
What it does: A frontier model — GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, Llama 3.1 at scale — arrives with an enormous range of capabilities already baked in from pretraining. Prompting shapes which of those capabilities activate and how the output is structured. A system prompt can establish a persona, constrain response format, set a tone register, provide task-specific context, and inject information the model wouldn't otherwise have access to. The model's weights don't change. Every call starts from the same base, and the input is the only variable.
Who's behind it / where it comes from: Frontier model providers — OpenAI, Anthropic, Google DeepMind, Meta AI — publish prompting guidance and system prompt documentation as part of their API offerings. The practice of structured prompting has been formalized through published research (OpenAI's system card documentation, Anthropic's model card and usage policies) and through community-developed frameworks like the DAIR.AI Prompt Engineering Guide, which synthesizes reproducible techniques across model families.
What makes it distinct: Zero retraining cost per iteration. You change the prompt, you change the behavior, immediately, with no GPU hours, no deployment pipeline, no model versioning. The tradeoff is that the model's knowledge is bounded by its training cutoff and whatever you inject into the context window at call time — prompting is fast to change and cheap to experiment with, but it doesn't expand what the model fundamentally knows.
Fine-Tuning
What it is: Continuing the training process on a pre-trained base model using a curated dataset, modifying the model's weights to shift its behavior in a target direction.
What it does: Fine-tuning takes a model that already knows how to do things and adjusts how it does them. You provide labeled examples — input-output pairs that demonstrate the behavior you want — and the training process updates the model's parameters to make that behavior more probable. The result is a model that produces outputs more consistently aligned with your examples: a particular writing style, a specific response format, a domain-specific vocabulary register, a tendency to follow certain structural patterns. What fine-tuning does not reliably do is inject new factual knowledge. Research from Zhu et al. (2023) and subsequent work on "knowledge editing" has consistently shown that fine-tuning on factual data produces models that are more likely to produce confident-sounding outputs in a domain without reliably producing accurate ones. The model learns the shape of your data, not the content.
PEFT and LoRA: Full fine-tuning — updating all of a model's parameters — is expensive. A 70-billion-parameter model requires substantial GPU infrastructure and days of compute. Parameter-efficient fine-tuning (PEFT) addresses this by modifying only a targeted subset of the model's weights. Low-Rank Adaptation (LoRA), introduced by Hu et al. at Microsoft Research in 2021, is the dominant PEFT technique: it inserts small trainable matrices into the model's attention layers, representing the adaptation as a low-rank decomposition of the weight update. In practice, LoRA fine-tunes roughly 0.1% to 1% of a model's parameters while achieving results comparable to full fine-tuning on style and task-format objectives. This makes fine-tuning accessible on single-GPU infrastructure for smaller models and dramatically reduces cost for larger ones. When a vendor says "fine-tuning," they may mean LoRA without saying so. Worth asking.
Who's behind it / where it comes from: Fine-tuning infrastructure is offered by frontier model providers (OpenAI's fine-tuning API for GPT-3.5 and GPT-4o mini, Google's Vertex AI tuning, Anthropic's fine-tuning program in limited access) and through open-source tooling built around Hugging Face's PEFT library and frameworks like Axolotl and LLaMA-Factory. The LoRA paper has over 10,000 citations as of early 2026 and is effectively the standard reference for efficient adaptation.
What makes it distinct: Fine-tuning produces a persistent model artifact — a set of weights, or a LoRA adapter — that encodes a behavioral pattern. That artifact can be deployed at lower inference cost than a large frontier model if you're using a smaller base model, and it tends to produce more consistent stylistic outputs than prompting alone because the behavior is in the weights, not dependent on the prompt being constructed correctly every time. The cost is upfront: compute, data curation, evaluation, and ongoing maintenance as base models update.
Comparison Strategy: Trait-Led Analysis
I'm using trait-led analysis here rather than scenario mapping or clustering because the executive misconception this piece is correcting lives on a specific axis — knowledge — and the most useful thing I can do is show the reader exactly where each approach sits on four decision-relevant dimensions. Scenario mapping would let the misconception hide inside use-case framing. Trait-led analysis forces it into the open.
Knowledge
Prompting expands the model's effective knowledge at call time by injecting context into the input. The model can reason over documents, data, and instructions it receives in the prompt. That knowledge is ephemeral — it exists for the duration of the call. Fine-tuning does not reliably expand the model's factual knowledge. It shifts the probability distribution over outputs in ways that reflect the style and structure of your training data. If your fine-tuning dataset contains accurate domain facts, the model may reproduce them more fluently — but it will also reproduce errors more fluently. The model learns the pattern, not the ground truth.
The executive misconception lives on this axis. "We need to train our own model on our proprietary data" almost always means "we want the model to know things it doesn't currently know." Retrieval is the tool for that job: getting the right information into the context window at call time. Fine-tuning addresses a different requirement — consistent behavior regardless of who writes the prompt.
Style and Consistency
Prompting can produce consistent stylistic outputs, but consistency depends on the prompt being constructed correctly every time. Drift happens — across users, across sessions, across prompt versions. Fine-tuning encodes stylistic behavior in the weights, making it more robust to prompt variation. If your requirement is that every output follows a specific format, uses a particular vocabulary, or maintains a specific tone register across thousands of calls from different users, fine-tuning is structurally better suited to that requirement than prompting alone. Fine-tuning earns its cost here.
Cost
Prompting costs per call. Frontier model API pricing is typically per million tokens, and costs scale with context window size and model tier. There is no upfront investment beyond prompt development. Fine-tuning costs upfront — data curation, compute, evaluation — and then per call on whatever infrastructure hosts the fine-tuned model. For high-volume, low-context use cases, a fine-tuned smaller model can be cheaper per call than a frontier model with a large system prompt. For low-volume or rapidly-changing use cases, the upfront cost of fine-tuning rarely recovers. The crossover point depends on volume, model size, and how often the behavioral target changes.
Latency
Frontier models are large. Inference on a 70B+ parameter model takes time, and API latency includes network overhead. A fine-tuned smaller model — a 7B or 13B base with a LoRA adapter — can produce outputs faster and at lower per-call cost, with the tradeoff that its general capability ceiling is lower. If your use case is narrow enough that a smaller model can handle it reliably, fine-tuning that smaller model and deploying it on dedicated infrastructure can produce latency profiles that frontier model APIs can't match. Real-time applications feel the difference. Asynchronous workflows mostly don't.
Field Language Guide
| Don't say | Do say | Why it matters |
|---|---|---|
| "We need to train our own model" | "We need to fine-tune a base model for this use case" | "Train our own model" implies building from scratch; buyers who hear that expect a multi-year research program |
| "Fine-tuning teaches the model our data" | "Fine-tuning adjusts how the model responds, not what it knows" | The knowledge framing is the core misconception; correcting it early prevents misaligned expectations |
| "Prompt engineering" | "System prompt design" or "inference-time instruction" | "Engineering" sounds like a workaround; "system prompt design" is the accurate term for a legitimate architectural decision |
| "The model will learn our policies" | "The model can be tuned to follow the format our policies require" | Policies are facts and logic; fine-tuning handles format and style, not rule-following at the semantic level |
| "Custom model" | "Fine-tuned model" or "adapted model" | "Custom" implies bespoke capability; "fine-tuned" accurately describes the relationship to the base model |
| "Inference cost" | "Per-call cost" | Buyers outside ML engineering don't use "inference" as a budget line; "per-call" maps to how they think about API pricing |
| "Base model" | "Foundation model" | Both are used; "foundation model" is the term that appears in OMB and NIST documentation, which is what federal buyers are reading |
| "PEFT" (unexplained) | "Parameter-efficient fine-tuning — a method that modifies a small fraction of the model's weights instead of all of them" | Acronym-dropping without translation signals vendor fluency theater; the explanation takes five seconds and builds credibility |
| "LoRA adapter" (unexplained) | "A LoRA adapter — a compact set of modifications layered onto the base model, like a configuration overlay" | The analogy gives buyers a mental model without requiring them to understand matrix decomposition |
| "The model hallucinates because it doesn't know our data" | "The model needs access to your data at call time — that's a retrieval problem, not a tuning problem" | Separates the diagnosis from the wrong prescription before the wrong prescription gets into the SOW |
| "Fine-tuning is more powerful" | "Fine-tuning is more consistent for style; prompting is more flexible for knowledge" | "More powerful" is undefined and invites the wrong comparison; the axes are what matter |
| "We can fine-tune it to stop hallucinating" | "Fine-tuning can reduce certain error patterns, but it doesn't solve factual accuracy — that requires grounding the model in verified sources at call time" | Sets accurate expectations before a pilot fails and the vendor takes the blame |
Okta Concept Mapping
The analog: Think of system prompt design as Okta's inline hooks — runtime instructions that shape how a transaction is handled without modifying the underlying policy configuration. Fine-tuning maps closer to modifying a custom authorization server's default behavior at the configuration level: the change is persistent, compiled into the system, and applies across all subsequent calls without re-instruction.
Where it holds: Both distinctions capture the runtime-vs.-persistent axis accurately. A system prompt, like an inline hook, is evaluated fresh on every call. A fine-tuned model's behavioral adjustments, like a modified authorization server policy, are always present.
Where it breaks: In Okta, modifying an authorization server's configuration genuinely changes what users can access — the scope of the change is real and consequential at the permission level. Fine-tuning modifies how the model responds: its behavior, not its access scope or factual knowledge. The Okta analog would suggest that fine-tuning expands the model's capability the way a policy change expands access scope. It doesn't. Fine-tuning changes behavior within the model's existing capability envelope. That distinction is exactly what the executive misconception gets wrong, and it's worth naming explicitly if the analog comes up in a buyer conversation.