Short Answer: What is chain-of-thought prompting?
Chain-of-thought prompting asks a large language model to work through intermediate steps before giving a final answer. Making the reasoning explicit improves accuracy on multi-step problems and lets you see which step went wrong when the answer is off. It is a way of writing prompts, not a change to the model.
Quick Summary
- Chain-of-thought prompting means asking a model to reason step by step before it answers.
- It helps on multi-step work: arithmetic, logic, commonsense reasoning, planning, and structured analysis.
- CoT is a prompting strategy, not an architecture change. It works on existing models with no retraining.
- Gains are measurable. Self-consistency adds 17.9 percentage points on the GSM8K math benchmark over standard CoT.
- In 2026 practitioners combine CoT with self-consistency, tree-of-thought, verification, and tool use.
What does chain-of-thought prompting mean?
Chain-of-thought prompting is a prompt engineering method that improves reasoning by asking a large language model to break a problem into intermediate steps written in plain language, before it commits to an answer.
A chain of thought is the sequence of reasoning steps between a question and its answer. Writing those steps out makes the path visible, so you can inspect how the model got where it did.
It mirrors how people handle hard problems: break them into smaller ones rather than leaping to a conclusion. Asking a model to explain its reasoning turns what would otherwise be a compressed, opaque prediction into a visible chain of logic.
CoT is a prompting strategy, not an architecture change. You can apply it to any current frontier model, whichever you happen to be using, without touching weights or retraining.
The difference from a standard prompt is small on the page and large in the output. A standard prompt asks, “What is 47 × 83?” A chain-of-thought prompt asks, “What is 47 × 83? Reason step by step before answering.” That single instruction changes output quality on anything with more than one moving part.
How does chain-of-thought prompting work?
CoT gets the model to decompose a problem inside a single prompt. The model generates a reasoning trace, a sequence of intermediate conclusions that builds toward the answer, and each step becomes context for the next.
A typical workflow:
- You write a prompt containing a CoT instruction or worked examples.
- The model generates intermediate reasoning steps, each building on the last.
- The model produces a final answer grounded in those steps.
- You read the chain to verify the answer or find where it went wrong.
The Prompting Guide’s chain-of-thought overview collects further pattern examples if you want templates to adapt.
What are the main types of chain-of-thought prompting?
Three ways to implement it.
| Implementation mode | How it works | Best for |
|---|---|---|
| Zero-shot CoT | Add an instruction such as “let’s think step by step,” with no worked examples | Rapid experimentation, low-stakes tasks |
| Few-shot CoT | Provide example problems with their reasoning steps for the model to follow | High-accuracy tasks where you can supply good examples |
| Auto-CoT | The model generates its own sub-questions or reasoning plan first | Domain-specific work where writing examples by hand is impractical |
Zero-shot is the easiest to try. Few-shot is usually better when accuracy matters, because the examples show the model which reasoning pattern to imitate. Auto-CoT earns its place when manual example-writing is too expensive or the task spans many domains.
How is CoT different from few-shot prompting, prompt chaining, and tree-of-thought?
Ordinary few-shot prompting gives the model input-output pairs. CoT examples include the reasoning steps between them, which is the part that transfers.
Prompt chaining splits a task across several separate prompts or API calls. CoT does its reasoning inside a single generation.
Tree-of-thought extends CoT by exploring several reasoning branches before choosing one. CoT is a single path through a problem; tree-of-thought is a map of the possible paths.
What are the benefits of chain-of-thought prompting?
The gains show up on work that takes multiple steps, not on factual recall.
- Better accuracy on multi-step tasks. Wei et al. showed that prompting a 540-billion-parameter model with eight chain-of-thought examples reached state-of-the-art accuracy on GSM8K math word problems, beating a fine-tuned model with a verifier.
- Transparency. A visible reasoning path means that when the answer is wrong you can find the step that broke, instead of re-prompting blindly.
- Easier human verification. Showing the reasoning lets a reviewer check the logic rather than trusting the conclusion, which matters most under time pressure.
- Breadth. It works across arithmetic, commonsense reasoning, symbolic reasoning, planning, and structured decisions.
- Decomposition. Forcing a split into sub-tasks extends the range of problems a model handles reliably.
- Fewer confident errors. On open-ended questions, examining intermediate assumptions reduces the confident-but-wrong pattern that direct prompting encourages.
For teams running high-stakes workflows, an inspectable reasoning chain doubles as an audit trail. When a system can show why it recommended something, the people accountable for the decision are more willing to act on it.
What are examples of chain-of-thought prompting?
Three worked examples, in increasing order of how badly direct prompting handles them.
An arithmetic word problem
Prompt
A store sells notebooks for $4 each and pens for $2 each. Maria buys 3 notebooks and 5 pens, then uses a $5 discount coupon. How much does she pay? Think step by step.
Expected response
Step 1: Cost of notebooks = 3 × $4 = $12
Step 2: Cost of pens = 5 × $2 = $10
Step 3: Total before discount = $12 + $10 = $22
Step 4: Apply $5 coupon = $22 − $5 = $17
Maria pays $17.
Without the instruction, models skip the discount or fumble the subtotal. The explicit chain makes each stage checkable.
A commonsense question
Prompt
If I put a frozen pizza in a hot oven and leave for 3 hours, what will I find when I return? Let’s think step by step.
Expected response
Step 1: A frozen pizza takes roughly 12–18 minutes at oven temperature.
Step 2: After 18 minutes it would be fully cooked.
Step 3: Three hours means it keeps heating for about 2.5 hours past that.
Step 4: It will be severely overcooked, likely charred, possibly smoking.
You will find a burnt pizza and possibly a smoke-filled kitchen.
A direct prompt tends to answer “a cooked pizza.” The chain forces the model to reason about duration and consequence.
A logic puzzle
Prompt
There are three boxes. One contains only apples, one only oranges, and one both. All labels are wrong. You pick one fruit from the box labeled “Apples and Oranges.” It’s an apple. What do all three boxes actually contain? Let’s think step by step.
Expected response
Step 1: All labels are wrong, so “Apples and Oranges” does not contain both.
Step 2: I drew an apple, so it must be the apples-only box.
Step 3: The box labeled “Oranges” cannot hold only oranges, and cannot hold only apples, so it holds both.
Step 4: The box labeled “Apples” must hold only oranges.
The labels are designed to mislead, which is why direct prompting struggles. CoT lets the model eliminate possibilities in order.
How CoT compares with a standard prompt
| Approach | Prompt | Typical output |
|---|---|---|
| Standard | “What is 15% of 240 plus 18% of 150?” | “63” |
| Chain-of-thought | “What is 15% of 240 plus 18% of 150? Show your reasoning.” | “15% of 240 = 36. 18% of 150 = 27. Total = 36 + 27 = 63.” |
Both reach the same answer. Only one lets you confirm the arithmetic without redoing it.
Why does chain-of-thought prompting improve reasoning?
It gives the model room to process each sub-problem before committing. Each generated step functions as additional working space inside the context window.
Three mechanisms are doing the work:
- Decomposition. Complex problems become a series of manageable sub-tasks.
- Error localization. When a step is wrong, it can be identified and corrected.
- Context reinforcement. Each step feeds the next, keeping the relevant details in active context.
Worth knowing
A reasoning trace is not a window into the model’s internal computation. It is a surface artifact, useful for verification, but the displayed steps do not necessarily match what happened inside the model. Treat it as a checkable account, not a confession.
Results also scale with model capability. Larger models benefit more, because they can sustain longer reasoning without drifting. Smaller ones often produce intermediate steps that look plausible and are wrong, which is its own kind of hallucination problem.
What CoT techniques are used in 2026?
CoT is now one layer in a stack rather than the whole approach.
| Technique | Description | Best for | Complexity |
|---|---|---|---|
| Zero-shot CoT | “Think step by step” instruction | Quick experiments | Low |
| Few-shot CoT | Worked examples with reasoning | High-accuracy tasks | Medium |
| Self-consistency | Multiple chains, majority vote | Production reliability | Medium |
| Auto-CoT | Model-generated reasoning plan | Broad domain coverage | Medium |
| Tree-of-thought | Multi-branch exploration | Complex problem spaces | High |
| Hypothesis-of-thought | XML-grounded fact highlighting | Reducing hallucination | High |
| CoT + tools | External validation of steps | Mission-critical workflows | High |
Zero-shot CoT
Append a phrase such as “let’s think step by step” and the model reasons without seeing any examples. Kojima et al. established that this alone produces a measurable improvement. Best for quick iteration and low-stakes work.
Few-shot CoT
Give the model two or three worked examples including their reasoning, and it follows the pattern. This is the standard move when accuracy matters, and it improves reasoning without task-specific fine-tuning.
Self-consistency
Run the same prompt several times, then take the most common answer instead of trusting a single chain. Wang et al. report this adds 17.9 percentage points on GSM8K over standard CoT, alongside gains on SVAMP, AQuA, StrategyQA, and ARC-challenge. Worth the extra calls wherever reliability beats cost.
Auto-CoT
The model writes its own reasoning plan before answering, which removes the burden of hand-crafting examples. It still needs evaluating, because a generated plan can be wrong in ways a human-written one would not be.
Tree-of-thought
Tree-of-thought explores multiple branches before committing, using search strategies to navigate the space. Best on problems with several valid approaches: strategic planning, creative work, optimization.
Hypothesis-of-thought
HoT grounds facts with XML tags before reasoning over them, which reduces hallucination compared with plain CoT. It has been evaluated across more than 22 tasks spanning arithmetic, reading comprehension, and logic.
CoT with verification and tools
CoT gets more reliable when its steps can be checked by something other than the model: calculators, code execution, retrieval, structured validation. For high-stakes work, teams combine CoT with output-format constraints, self-checking, and tool calls, so intermediate conclusions are validated rather than trusted. That is the bridge to agentic AI workflows.
When should you not use chain-of-thought prompting?
On simple, single-step tasks, direct prompting is faster, cheaper, and just as accurate. “What is the capital of France?” does not need a reasoning chain; asking for one adds cost, latency, and tokens for nothing.
Use CoT where the path to the answer genuinely matters: math word problems, logic puzzles, planning, structured analysis, multi-variable decisions, and anything a person will need to audit. It also interacts with temperature, since self-consistency depends on sampling varied reasoning paths rather than the same one repeatedly.
Two things to watch: reasoning-optimized models increasingly do this internally by default, which makes the explicit instruction redundant in some systems, and teams running CoT in production are building monitoring and evaluation layers around it rather than treating the prompt as the whole solution.
Frequently Asked Questions
What is the difference between zero-shot and few-shot chain-of-thought prompting?
Zero-shot uses an instruction such as “let’s think step by step” with no examples. Few-shot includes worked problems with their reasoning so the model can follow the pattern. Few-shot is usually more reliable and takes more upfront effort.
When should chain-of-thought prompting be used?
For complex, multi-step problems where the path to the answer matters: math word problems, logic puzzles, planning, structured analysis, and reasoning-heavy workflows. For simple lookups, standard prompting is more efficient.
Does chain-of-thought prompting work with all large language models?
It can be applied to any of them without changing the architecture, but results vary with model capability. Gains are limited in smaller models and more pronounced in larger ones.
What are the main limitations of chain-of-thought prompting?
More tokens, higher latency, sensitivity to how the prompt is written, and the risk of reasoning that sounds plausible and is wrong. The visible trace also does not guarantee that the model’s internal process matches the written steps.
How can teams improve chain-of-thought reliability in production?
Clear prompt formatting, self-consistency, tool-based verification, human review, and automated evaluation. For high-stakes applications, combine CoT with agentic workflows and explicit checkpoints rather than relying on the reasoning trace alone.
About the author
Kai Williams
Kai Williams has been in marketing for years, with a long background in SEO before AEO had a name. He stepped into Answer Engine Optimization the moment AI started reshaping how people search, and has been tracking the shift ever since. At Prompt Insider, he covers AEO, AI marketing, and the future of search, breaking down what is changing and what brands need to do about it.
