System Prompt

Definition. The privileged instruction block sent before any user turn that defines the assistant's identity, scope, rules, tone, available tools and output contract. It is the closest thing an LLM application has to configuration, and in most architectures it is the only place where non-negotiable constraints live. Providers generally give it higher priority than user messages, but that priority is a learned behaviour, not an enforced boundary.

How it works. Write the system prompt as a specification document: purpose, audience, capabilities, hard rules, refusal policy, output format, and examples. Keep it stable across requests so it can be cached by the provider, which typically cuts input cost substantially. Store it in version control with an identifier, log that identifier with every request, and evaluate changes against a regression suite before rollout.

Example. A banking assistant's system prompt is 1,400 tokens and includes a rule that it must never state an account balance and must instead direct the user to the authenticated app. During a red-team exercise a user asks the assistant to role-play as a teller with no restrictions. Because the refusal rule is stated twice, once at the top and once in a closing constraints block, and because the application also filters the response, the attempt fails. The prompt alone would not have been sufficient.

Use case. The system prompt is where you encode brand voice, regulatory constraints, tool-use policy, output schemas and escalation rules. It is also the natural place to implement multi-tenant behaviour by templating tenant-specific sections while keeping a shared, cacheable core.

Warnings. Never place API keys, internal URLs or personal data in a system prompt; assume it can be extracted. Do not treat it as an access-control mechanism. And do not let it grow without bound: every token is billed on every request, and long prompts dilute attention.

Limitations. System prompts are vulnerable to injection and to instruction dilution as they grow. Their effect varies by model and can regress silently when a provider updates the model. They also cannot enforce anything: enforcement must happen in code around the model.

← Browse all glossary terms