Prompt Engineering

Every LLM does exactly one thing at inference time: it predicts the most likely next token given every token that came before it. The prompt is the “everything that came before” — the full context from which the model generates its response.

This means the prompt is not a command in the traditional software sense. It is a conditioning signal. You are not telling the model what to do. You are shaping the probability distribution over its next token. Change a single word and the distribution shifts. Add three examples and it shifts dramatically.

A modern production prompt has several distinct components:

• System instruction — sets the model’s role, constraints, and behavioral guidelines
• Context / background — domain knowledge, retrieved documents, or conversation history. This is where RAG injects external knowledge.
• Examples — input-output pairs that demonstrate the desired behavior
• User query — the actual question or task
• Output format constraint — specifies JSON, bullet points, or a particular structure

Zero-shot means providing no examples at all. You rely entirely on the model’s pre-training and fine-tuning to understand your request.

Few-shot prompting — pioneered by the GPT-3 paper (Brown et al., NeurIPS 2020) — includes 2 to 5 input-output examples. This is in-context learning: the model adapts its behavior without any weight updates, purely from the examples in the prompt. It is the single highest-ROI prompting technique available.

Many-shot prompting takes this further. Google DeepMind’s research demonstrated that performance scales log-linearly with examples — on MATH, many-shot improved accuracy by 35% over few-shot (Agarwal et al., NeurIPS 2024).

What happens when the task requires actual reasoning? Simply asking for the answer often produces the wrong one. Chain-of-thought (CoT) prompting solves this by forcing the model to show its work.

The technique was introduced by Wei et al. (NeurIPS 2022). On GSM8K, PaLM 540B went from 18% with standard prompting to 57% with CoT — a 3× improvement from nothing more than including reasoning steps. Why? During inference, each output token is conditioned on all previous tokens. The intermediate steps become computational workspace.

The system prompt is the command layer of your prompt architecture. It sets the behavioral frame for the entire conversation. Four things go in:

1. Role definition — who the model should behave as
2. Behavioral constraints — what the model should and should not do
3. Output format — the structure of the response
4. Safety boundaries — guardrails for harmful or off-topic content

Does persona engineering actually work? Research offers a nuanced answer. Personas do not reliably improve performance on objective tasks — they explain less than 10% of annotation variance (Zheng et al., 2023). Where personas genuinely help is in shaping output style and scope. The more powerful technique is constraint engineering: specific, testable rules the model can follow or violate in observable ways.

Production systems need structured data — JSON objects, function calls, API payloads. Three approaches have emerged:

Structured outputs constrain at the API level. OpenAI’s Structured Outputs (August 2024) achieve 100% schema adherence — every response matches the provided JSON schema exactly.

Function calling lets the model select and parameterize tools from a provided catalog. The model generates a structured request; your code executes it. This separation is fundamental to building reliable systems.

Constrained decoding operates at the inference engine level, masking invalid tokens at each generation step. Modern engines compute masks in ~50 microseconds (vLLM, 2025) — the model physically cannot output invalid JSON.

Prompt engineering has failure modes as predictable as any other engineering discipline. The most dangerous is prompt injection — #1 in the OWASP Top 10 for LLM Applications, with attack success rates of 50–84%.

The best current defense is instruction hierarchy — training models to prioritize instructions by trust level (Wallace et al., ICLR 2025). But no defense is complete — a 2025 study bypassed all eight tested defenses with adaptive attacks (NAACL 2025). The practical implication: never trust LLM output in security-sensitive contexts without validation.