Chain-of-Thought (CoT) & Tree of Thoughts

1. The Computational Bottleneck: Why LLMs Need Tokens to "Think"

Consider a multi-step logic puzzle:

If you demand: "Answer with just the number:", the model must predict the token in one single pass through its layers. It frequently misfires with "8" (doing 17 - 9). However, if the prompt instructs: "Think through the problem step by step before stating the answer:", the model writes:

By generating the explanation tokens first, those tokens enter the model's in-context attention memory as inputs for subsequent tokens. The model effectively uses its own generated output as external working RAM!

2. Zero-Shot CoT vs. Few-Shot CoT

3. Self-Consistency: Majority Voting Across Reasoning Paths

Wang et al. (2022) observed that complex reasoning tasks often possess multiple valid reasoning paths that lead to the same correct solution. In Self-Consistency, you sample N independent CoT chains at temperature = 0.7 and aggregate answers via majority voting:

import collections
from openai import OpenAI

client = OpenAI()

prompt = """
Q: Janet's ducks lay 16 eggs per day. She eats 3 for breakfast every morning 
and bakes muffins with 4 every day. She sells the remainder at the farmers' 
market for $2 per egg. How much does she make each day?
Think step-by-step and end your response with: 'Final Answer: $<amount>'.
"""

def extract_answer(text: str) -> str:
    # Extracts the final numerical dollar value
    if "Final Answer:" in text:
        return text.split("Final Answer:")[-1].strip().replace("$", "")
    return "UNKNOWN"

# Sample 5 diverse reasoning chains in parallel
responses = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": prompt}],
    temperature=0.7,
    n=5  # Generate 5 candidate completions simultaneously
)

answers = [extract_answer(choice.message.content) for choice in responses.choices]
vote_counts = collections.Counter(answers)
majority_winner, count = vote_counts.most_common(1)[0]

print(f"Sampled Answers: {answers}")
print(f"Majority Consensus Winner: USD {majority_winner} ({count}/5 votes)")

4. Tree of Thoughts (ToT): Search with Backtracking

For non-linear tasks (such as solving the 24 Game, theorem proving, or architectural design), Chain-of-Thought can make an early flawed assumption and stubbornly follow it to an incorrect conclusion.Tree of Thoughts (Yao et al., 2023) maintains a tree of candidate intermediate thoughts:

5. The Modern Frontier: Built-in Reasoning Models (OpenAI o1 & o3)

With the introduction of models like OpenAI's o1 and o3, Chain-of-Thought has moved from developer prompt engineering into the model's fundamental training loop via Large-Scale Reinforcement Learning. These models generate hidden reasoning tokens, evaluating hypotheses, identifying errors, and backtracking autonomously before delivering the user-visible answer.