Lesson 2 of 23 ยท Part 0: Foundations

Temperature & Creativity

How one number controls whether the AI is a robot or a poet.

Last time: An LLM writes by guessing the next token, one at a time, from a list of chances.
Today: One dial, temperature, decides how boldly the AI picks words: steady robot or wild poet.
01.02.0

temperature = 0.0 โ†’ 2.0

๐Ÿค–Predictable
โš–๏ธBalanced
๐ŸŽฒCreative

"The capital of France is ___"

Model outputs probabilities for every possible next word:

Paris
92%
๐Ÿ‘ˆ
Lyon
3%
Marseille
2%
the
1.5%
known
1.5%

Temperature decides HOW to pick from these probabilities

The Math: Softmax Scaling

Try another:
P(token) = exp(logit / T) / ฮฃ exp(logits / T)

T = 0.1

Paris
>99.9%
Lyon
<0.1%
Marseille
<0.1%
the
<0.1%
known
<0.1%

Extremely peaked

T = 2.0

Paris
63%
Lyon
11%
Marseille
9.3%
the
8.1%
known
8.1%

Much flatter

Lower T โ†’ sharper peaks. Higher T โ†’ flatter distribution. (Illustrative scores; the formula is real.)

๐Ÿค–

The Robot

Temp = 0

Try another:

"The capital of France is ___" โ€” selection chances:

Paris
100%
ALWAYS โœ“
Lyon
Marseille
the
known

Always picks the #1 word. 100 runs โ†’ 100 identical answers.

Top word "Paris" gets picked about 100% of the time here (illustrative chances, real formula).

โš–๏ธ

The Sweet Spot

Temp = 0.7

Try another:

"The capital of France is ___" โ€” selection chances:

Paris
98%
Lyon
0.7%
Marseille
0.4%
the
0.3%
known
0.3%

Usually picks top words but sometimes surprises. A common choice for chat.

Top word "Paris" gets picked about 98% of the time here (illustrative chances, real formula).

๐ŸŽฒ

The Wild Card

Temp = 1.5

Try another:

"The capital of France is ___" โ€” selection chances:

Paris
76%
Lyon
7.8%
Marseille
6.0%
the
4.9%
known
4.9%

Wild and unpredictable. Low-probability words get a real chance.

Top word "Paris" gets picked about 76% of the time here (illustrative chances, real formula).

Prompt: "How do I sort a list in Python?"

Try another:

Same model, same prompt โ€” three temperatures (illustrative responses):

T=0
def sort_list(lst):
return sorted(lst)
ย 
# Simple, correct, identical every run.

Same output every time. Perfect for code.

T=0.7
"Great question! You can use Python's built-in sorted() function, or for in-place sorting, try lst.sort(). Here's an example with a custom key..."

Helpful, varied phrasing. Good for explanations.

T=1.5
"Ah, sorting! The eternal dance of data finding its place. Imagine each element as a restless traveler seeking its destined position in the grand array of existence..."

Wild tangent. Fun for stories, bad for code.

Top-P (Nucleus) Sampling

top_p = 0.9 โ€” keep the fewest top words whose chances add up to at least 90%

Paris
92%
IN
Lyon
3%
CUT
Marseille
2%
CUT
the
1.5%
CUT
known
1.5%
CUT

Temperature

Flatten the curve

Top-P

Cut the tail

Alternative to temperature: cut off low-probability tokens entirely.

Temperature โ‰  Quality

Low Temp (0)

High Temp (1.5)

Good Prompt

Correct, concise answer

โœ“

Creative, still useful

โœ“

Bad Prompt

Confidently wrong

โœ•

Creatively wrong

โœ•

Prompt quality matters MORE than temperature

Guess the Temperature

Prompt: "Write a haiku about coding"

Response A

Code compiles and runs
Output matches expected
Ship to production
T=0

Response B

Fingers dance on keys
Logic weaves through lines of code
Bugs become features
T=0.7

Response C

Electric whispers
Through silicon dreams we float
Moonlight types itself
T=1.5

API Differences

Provider

Range

Default

OpenAIOpenAI

0 โ€“ 2

1.0

AnthropicAnthropic

0 โ€“ 1

1.0

GeminiGemini

0 โ€“ 2

(varies)

Same concept, different scales. Always check the docs.

Match temperature to your task

๐Ÿ’ป

Precise

0 โ€“ 0.3

โ€ขCode generation
โ€ขMath problems
โ€ขData extraction
โ€ขFactual Q&A
๐Ÿ’ฌ

Balanced

0.5 โ€“ 0.8

โ€ขGeneral chat
โ€ขSummaries
โ€ขExplanations
โ€ขEmail drafting
๐ŸŽจ

Creative

1.0 โ€“ 1.5

โ€ขCreative writing
โ€ขBrainstorming
โ€ขPoetry
โ€ขCharacter dialogue

Key Takeaways

๐ŸŽ›๏ธTemperature controls randomness in token selection (0 to 2)
๐Ÿค–Temp 0 = deterministic โ€” always the same output
โš–๏ธTemp ~0.7 = balanced โ€” creative but controlled
๐ŸŽฒTemp 1.5 = creative chaos โ€” surprising & unpredictable
๐Ÿ”ขSoftmax divides logits by T โ€” lower T sharpens, higher T flattens
โœ‚๏ธTop-P is an alternative โ€” cuts off low-probability tokens
๐ŸŽฏMatch temperature to your task โ€” code low, stories high
โ† Prev
1/0
Next: Context & Memory โ†’