cadence_learn
start_with_ai / level 09 of 10 · building ▶ 12 min

open the black box.

You've used the machine for eight levels. Now you get to see inside it. Not the maths — the five ideas that explain the weird stuff you've probably already noticed: why long chats drift, why it can't say who won last night, why some models cost 100× others.

after this level you'll be able to
  • Explain model, training, context window, and token — in your own words
  • Fix the long-chat drift problem like someone who knows why it happens
  • Choose between fast-cheap and deep-smart models on purpose
idea 1 + 2

a model is frozen study

the mental model

Training is the studying: months of finding patterns in enormous amounts of text. The model is what's left after — billions of numbers encoding those patterns, frozen. When you chat, it isn't learning. It's applying.

This explains two things you've met already. Why the knowledge cutoff exists (Level 01): the studying stopped on a date; anything after isn't in the numbers — unless the app bolts on web search. And why it doesn't remember you between chats: talking to it doesn't change it. Nothing you type rewires the model. Each conversation starts from the same frozen snapshot.

idea 3

the context window — its working memory

The model's only "memory" of your conversation is the conversation itself — re-read in full, every single reply. The amount it can re-read at once is the context window. Long chats drift because the window fills — and the early stuff effectively falls off the table.

You've felt this: forty messages in, it forgets what you said at the start, contradicts itself, gets vaguer. Now you know why — and the fix is mechanical, not magical:

the fresh-start move — use this forever

When a long chat gets fuzzy: start a new chat and re-feed only what matters — "Here's where we got to: [3-line summary + the current draft]. Continue from here." Fresh window, zero clutter, all signal. This one habit fixes 90% of "the AI got dumb" complaints.

idea 4 + 5

tokens, and why price varies 100×

Tokens are how it reads: text chopped into chunks of a few characters. "Understanding" might be one token; "misunderstanding" three. This explains the small weirdness — counting letters in a word is genuinely hard when you can't see letters — and it's the billing unit: every token in and out is compute, and compute is money.

Model tiers follow directly: small models (fewer numbers) are fast and cheap and fine for everyday asks; big models are slower, pricier, and visibly better at hard reasoning. The free tier of most apps runs the smaller ones — which is why this course never needed the paid tier, and why the paid tier exists.

the practical rule

Everyday questions, summaries, drafts → any model is fine. Hard reasoning, long documents, code, anything where wrong is expensive → use the best model you can get, and give it room to think ("think step by step before answering" genuinely helps). Matching the model to the job is a builder's habit — most people never learn it.

do it now

watch the window fill

your turn 5 minutes
  1. Find your longest old chat — one that got fuzzy near the end. Scroll it and spot the drift: the point where it started forgetting or repeating. That's the window filling. It'll be obvious now that you know what you're looking at.
  2. Do the fresh-start move for real:
    in a NEW chatContinuing an earlier conversation. Here's everything that matters from it: [3 lines: what we were doing, what we decided, where we stopped] Pick up from there: [the next thing you want]
  3. Compare the sharpness of the reply to the fuzzy end of the old chat. Same model — empty window. Now the fix is yours for life.
level complete

three things to carry into level 10

01

Frozen study, applied

Training made the numbers; chatting doesn't change them. Cutoffs and blank slates both follow.

02

The window is the memory

Long chats drift because it fills. Fresh chat + re-feed the essentials = sharp again.

03

Match model to job

Cheap-fast for everyday, deep-smart for hard. Choosing on purpose is a builder's habit.

Stuck? Peter reads these personally and replies to your email.
Ask Peter →