AI Foundations · Lesson 4

What Is a Large Language Model?

See how language models predict text a piece at a time, why they can sound fluent, and why they still need human checks.

70 minBeginnerBusinessAdvanced

Progress is saved in this browser only.

Learning paths shown

The core lesson is always shown. Beginner is shown to start; tick Business or Advanced to add them, in any combination. Your choice applies to every lesson and is saved in this browser.

Introduction

Fluent language is not the same as a stored answer

If you have used a chat assistant, you have used a large language model, often shortened to LLM.

The experience can feel like talking to a knowledgeable person. That feeling is useful as motivation and dangerous as a theory of how the system works.

A language model is a neural network trained on text so that it can predict likely next pieces of language. When you see a full paragraph, you are seeing many of those predictions, one after another, shaped by your prompt and by the application around the model.

This lesson builds on Lesson 3. You already know that a neural network has parameters fitted from examples, and that a training score is not a guarantee. Here the "examples" are pieces of text, and the "output" is more text.

You do not need the internal architecture of any particular product. You need a mental model that keeps you from two mistakes:

  1. Treating the assistant as a person who understands your life.
  2. Treating the assistant as a database that only repeats verified facts.

By the end, you should be able to explain tokens, next-token prediction, and context in plain language, and you should know why Lesson 5 (prompt engineering) is worth practicing.

What you will learn

  • Explain a large language model as a neural network trained to work with language.
  • Describe tokens as the pieces of text a language model reads and writes.
  • Explain embeddings as learned lists of numbers that represent tokens.
  • Describe attention as the way a transformer model weighs other tokens in the context.
  • Describe generation as repeated next-token prediction rather than retrieval of a stored essay.
  • Identify what counts as context and what the model does not automatically know.
  • Distinguish training from inference, and the trained model from the chat product that wraps it.
  • Recognize why fluent language can still be incorrect, incomplete, or unsupported.
  • Choose tasks where a language model is a reasonable assistant and tasks that need exact software or a person.

The lesson

1. What a large language model is

A language model estimates patterns in language: which words, pieces of words, and punctuation tend to follow others in context.

A large language model is a language model with enough capacity and training to handle a wide range of wording, topics, and formats. This lesson uses "large" as a qualitative description. It does not quote parameter counts, dataset sizes, or training costs for any named product, because those figures change and are easy to get wrong.

Connect it to what you already know:

  • AI is the broad field.
  • Machine learning is learning patterns from data.
  • A neural network is one kind of model.
  • A large language model is a neural network specialized for language.

Not every AI system is an LLM. A model that only classifies Harborlight messages as billing, scheduling, or service can be machine learning without generating a paragraph. An LLM is built to produce language (and, in many products, to follow instructions expressed in language).

A useful beginner definition:

A large language model is a neural network trained on a large amount of text so that it can predict likely next pieces of language and, in a chat product, use that ability to draft, explain, summarize, or continue text.

The definition is about a method. It is not a claim that the model knows the truth, has a biography, or has live access to your files.

2. Tokens: the pieces the model actually uses

People read words and sentences. A language model reads and writes tokens.

A token is a chunk of text defined by a tokenizer, the component that splits text into the units the model was trained on. Depending on the tokenizer, a token might be:

  • A whole common word, such as the.
  • Part of a longer word, such as the pieces of scheduling.
  • Punctuation or a space.
  • In some systems, other kinds of markers.

You do not need to memorize a vendor's vocabulary. You need the practical consequences.

Why tokens matter

Length is measured in tokens, not pages. A product's context limit is a token budget. A short prompt with a pasted contract can still be large.

Spelling and format affect tokens. Extra spaces, unusual punctuation, or mixed languages can change how text is split.

A PDF reaches the model through the application, not as the page you see. Many applications extract the PDF's text first and send that text as tokens. If the extraction is messy (scanned pages, columns, tables, headers, or footnotes), the tokens are messy. Some multimodal systems can also process page images directly, turning them into model inputs alongside or instead of extracted text. Either way, the model works with a converted version of the document, so check tables, numbers, and anything that appeared only in an image.

A small illustration

The sentence I need Friday off. is a handful of words for you. For a model it is a sequence of tokens. Another sentence with the same meaning, Please approve leave on Friday., is a different sequence. The model can still relate them if training taught it that those patterns are similar, but it is not matching your intent by magic. It is processing token sequences.

When a response stops mid-sentence, a common reason is that an output token limit set by the application or service was reached. It can also happen because the conversation filled the context window, or because the application ended generation for another reason, such as a stop rule, a timeout, a dropped connection, or a safety filter. The interface does not always say which, so if an answer looks unfinished, ask it to continue or shorten the request.

3. Embeddings: how tokens become numbers

A neural network works with numbers, not letters. So after the tokenizer splits text into tokens, two more steps turn each token into something the network can calculate with.

Step 1: a token ID. Each token in the tokenizer's vocabulary has an ID number. The ID is only a label, like a locker number. Token 4012 is not "bigger" or "more important" than token 17.

Step 2: an embedding. The model looks up a learned list of numbers for that ID. This list is called an embedding (or embedding vector). It is the token's starting representation inside the network.

You can think of an embedding as coordinates on a map. On a paper map, two numbers place a town. An embedding uses many more numbers, so it can place a token in a space with many directions at once. Tokens that are used in similar ways in the training text tend to end up with embeddings that are near each other in that space. Nobody types these numbers in by hand: they are learned during training, along with the rest of the model's parameters.

A worked example with made-up numbers

The table below is illustrative only. Real tokenizers produce different IDs, and real embeddings use far more numbers per token. The point is the shape of the process.

TokenToken ID (made up)Embedding (made up, 3 numbers)
heater51200.82, 0.10, 0.33
furnace90770.79, 0.14, 0.30
invoice22310.05, 0.91, 0.12

In this toy picture, heater and furnace sit close together because they show up in similar sentences, while invoice sits somewhere else.

Order matters too

"The dog bit the man" and "The man bit the dog" contain the same tokens. To tell them apart, transformer models also include information about each token's position in the sequence, combined with its embedding.

Where you may meet embeddings again

Embeddings are also used outside chat. Many search and "retrieval" features turn documents and questions into embeddings and look for ones that are close together. That is one way an application finds relevant passages to place in the model's context, which section 6 returns to.

4. Attention: how the model uses the rest of the context

Many current large language models, including those behind popular chat assistants, are built on a neural-network design called the transformer. Its central mechanism is attention.

Attention answers a practical question for every token: which other tokens in the context should shape my representation right now, and how much?

A sentence that needs attention

The technician fixed the heater because it was broken.

To understand what "it" refers to, a reader looks back at "heater" rather than "technician." Attention gives the model a learned way to do something similar: when it builds the representation for "it," it can draw more heavily on some earlier tokens than others.

The weights below are made up to show the idea. Real models compute their own weights, and they are not this tidy.

When processing "it", how much to draw on...Illustrative weight
technician0.10
fixed0.05
heater0.70
because0.05
other tokens0.10

The weights add up to 1. The model blends information from the earlier tokens according to those weights, so the representation of "it" ends up carrying information from "heater."

Layers and heads

A transformer repeats this step many times. Each layer contains several attention calculations side by side (often called heads), and different heads can pick up different kinds of relationships, such as which word a pronoun refers to or which words belong to the same phrase. Layer after layer, each token's representation is refined using the context around it.

Two limits to remember
  • When generating text, a language model attends only to tokens that are already in the context. It cannot look at words it has not written yet, or at information that was never provided.
  • Attention weights are not a full explanation. They show part of what the model computes, not a reliable account of why it produced a particular answer.

Attention is why a long, specific prompt can steer an answer: every token you provide is available for the model to draw on.

5. Next-token prediction

Training a language model is, at its core, practice at a prediction task: given the tokens so far, estimate what is likely to come next.

During generation (when you receive an answer), the system:

  1. Takes the current context as a token sequence.
  2. Turns the tokens into embeddings and runs them through the model's attention layers.
  3. Asks the model for a distribution over possible next tokens.
  4. Chooses a next token (sometimes the most likely one, sometimes a sample from the distribution).
  5. Appends that token to the context.
  6. Repeats until it hits a stopping rule: an end marker, a token limit, or an application-defined stop.

For example, after the context Please confirm your appointment on, a model might assign high probability to tokens such as Friday or Monday, and much lower probability to unrelated tokens. Whichever token is chosen is added to the context, and the whole calculation runs again for the token after it.

That loop is how a paragraph appears. The model is not selecting a pre-written article from a shelf labeled with your question.

What this explains

Variation. If the application samples from the distribution, two runs can differ.

Continuation. If you say "finish this list," the same mechanism continues the pattern.

Invention. If the context does not contain the fact you want, the model can still emit tokens that look like a fact, because its job is to continue language, not to refuse whenever it is unsure.

Local coherence. Each step is about a likely next token given recent context. A paragraph can be locally smooth and globally wrong.

A tiny character-level toy in the advanced companion makes this visible: even a small table of "what character tends to follow what" can produce text that looks a bit like the training snippet and is not a reliable explanation of the world.

6. Context: what the model can use right now

Context is the information available for the current prediction. In a chat product that usually includes:

  • Your latest message.
  • Earlier turns the application still includes.
  • Hidden instructions the product adds (often called a system prompt).
  • Any documents, tool results, or retrieved notes the application attached.

Context is limited. Products describe this as a context window: a maximum number of tokens of input (and, depending on the system, output) the model can consider at once.

When the window is full, older turns may be dropped, summarized, or otherwise reduced. The model does not secretly remember a conversation that is no longer in context.

Context versus stored information

It helps to separate four places information can live:

Where it livesWhat it isCan the model use it in this answer?
The model's parametersPatterns learned during trainingYes, but only as patterns. It cannot look up or quote a specific source reliably.
The context windowEverything sent with this one requestYes, directly. This is the only material the model reads for this answer.
Conversation historyEarlier turns the application keepsOnly the turns the application resends in the context. Long chats may be shortened or summarized.
Application memory and retrievalSaved notes, uploaded files, company documents, databasesOnly the parts the application finds and inserts into the context.

Application memory is a product feature: the application saves some information (for example a preference you stated) and adds it to later conversations. Retrieval means the application searches a store of documents, often by comparing embeddings, and inserts the passages it finds. Both happen in the application, not inside the model, and both end the same way: the information has to be placed into the context to affect the answer.

Stored is not the same as used. If the application does not retrieve something, the model cannot use it, and what it does retrieve can be incomplete or out of date. Using a chat does not change the model's trained parameters.

What is not automatically in context
  • Your private files, email, and calendar, unless you or an authorized integration provided them.
  • A company's live inventory, prices, or ticket status.
  • "Whatever is true on the internet today," unless the application actually retrieved something and you still have to check it.
  • The model's training data as a searchable library you can cite by default.

If you ask, "What did we decide last month?" and last month is not in the prompt, the model may guess. That guess can look like a memory.

A practical habit

When the answer depends on a fact, put the fact in the message or attach an approved source. Then ask the model to use that source and to mark anything it cannot find.

Lesson 5 will turn this habit into prompt patterns. The reason the pattern works is in this lesson: missing context becomes invented continuation.

7. Training, inference, and the chat product

Training versus inference

Two very different activities are both described as "the model learning" or "the model thinking." Keep them apart:

  • Training builds the model. The developer runs a long, expensive process over a large collection of text, adjusting the model's parameters to improve next-token prediction (and, later, to follow instructions). Training happens before you use the model.
  • Inference uses the model. When you send a prompt, the trained parameters stay fixed. The system runs the embedding, attention, and prediction steps to produce tokens for your answer.

Your conversation does not retrain the model as you type. Some providers may use conversations to help train future model versions, depending on their policies and your settings, but that happens later, separately, and produces a new version. Within your chat, anything that seems "remembered" comes from the context and the application, as section 6 describes.

Three layers in a chat product

It helps to separate three layers.

The pretrained language model

This is the neural network trained to model language, typically with a next-token objective on a large text collection. The details of that collection vary by developer and version. Do not assume you know every source, or that the model can quote those sources on request.

Later alignment or instruction training

Many assistants are further trained or adjusted so they more often follow instructions, refuse some categories of request, or use a preferred style. This is still learning from data and feedback. It is not a personality and it is not a guarantee.

The application

The product you open in a browser or app is software around the model. It may:

  • Add instructions you never see.
  • Truncate or format history.
  • Call tools (search, retrieval, calculators, company systems) if those tools are connected.
  • Filter or reroute some outputs.
  • Log the exchange according to its policies.

Two products can use similar model families and behave differently because the application layer differs. One product's browsing, file access, or "memory" feature is not a property of every LLM.

Lesson 2 still applies: the model proposes language. Ordinary software should enforce permissions. People remain accountable for actions that matter.

8. Why fluent answers can still be wrong

A language model is rewarded, during its core training, for continuations that look like language. Fluency is the skill it is built for. Truth is a different skill, and it is only as good as the context, the training data, the application checks, and your review.

Familiar failure modes

Unsupported details. Names, dates, policy clauses, and citations can be assembled because they fit the sentence, not because a source was checked.

Mismatched context. The model answers a nearby question, mixes two Harborlight policies you pasted, or ignores a constraint at the bottom of a long prompt.

Out-of-date or missing facts. Parameters do not automatically update when a price list changes.

Bias and tone problems. Patterns in training data and instructions can show up as unfair, stereotyped, or off-brand language.

Overconfidence. The sentence may not include "I might be wrong" unless the application or the prompt encourages that.

These are reasons to verify important results. They are not reasons to avoid LLMs for drafting, outlining, explaining, or exploring ideas.

A working rule

If you would not send the text without checking it if a new intern wrote it, do not send it without checking it because a model wrote it.

High-impact areas still need qualified people: medical, legal, financial, employment, safety, and anything that commits your business.

9. When an LLM is a good assistant

Language models tend to help when the job is to transform, draft, or organize language you can check.

Good starting tasks:

  • Turn rough notes into a cleaner outline or checklist.
  • Rewrite a paragraph for a stated audience, then compare it with the original.
  • Suggest questions to ask before a meeting.
  • Explain a concept at a stated level, which you then verify.
  • Produce alternative phrasings of text you already consider correct.

Weaker or unsafe starting tasks:

  • Exact arithmetic that must be right (use a calculator or tested software).
  • Live prices, schedules, or inventory without an authorized source.
  • Legal or medical conclusions.
  • Anything that requires sending confidential customer data to an unapproved tool.

Harborlight Home Services could ask an LLM to draft a reply from an approved FAQ, with a person sending it. Harborlight should not ask an LLM to invent a warranty decision.

The model is a language engine. Pair it with sources, rules, and review, and it becomes a tool. Leave those out, and you have a fluent guesser.

10. A bridge to prompt engineering

Because generation is next-token prediction in context, the instructions you write are part of the context. Clear tasks, constraints, audience, format, and source text change the distribution of likely continuations.

That is why a vague prompt such as "write a policy" produces generic text, and a prompt that includes the real constraints produces something you can edit.

Lesson 5 is devoted to that craft: role, task, context, format, examples, and iteration. This lesson is the mechanism underneath the craft.

You already practiced a small version in Lessons 1 and 2 when you compared a weak prompt with a strong one. After this lesson you should see those comparisons as context design, not as magic words.

Lesson takeaway

A large language model predicts tokens. A chat product wraps that model in software. Useful work happens when you supply the right context and check the language that comes out.

Do not look for a mind inside the network. Look for a completion engine, then use it with the same care you would use with any powerful draft.

Try it

Model, tokenizer, context, and application

Keep these four pieces distinct when you evaluate a chat tool.

PieceWhat it doesWhat it does not do by itself
TokenizerSplits text into tokens the model can process.Understand your goal as a person would.
Language modelEstimates likely next tokens given context.Guarantee facts, live data, or approved actions.
Context windowLimits how much tokenized text is available now.Store your whole work history forever.
ApplicationAdds instructions, history, tools, and controls.Make every generated sentence true.

Missing facts become invented continuation

The weak prompt asks for a specific business fact with no source. The strong prompt supplies the only facts that may be used.

Weaker prompt

What is Harborlight Home Services' weekend emergency fee?

Stronger prompt

Using only the policy text below, answer the customer's question: 'Do you charge extra on weekends?' If the policy does not say, reply that the fee is not listed and a person must confirm. Do not invent a dollar amount.

Policy: Harborlight Home Services schedules weekday visits from 8:00 to 17:00. Weekend visits require manager approval. Fees are quoted per job and are not listed in this policy.

The weak prompt invites a fluent guess. The strong prompt puts the allowed facts in context and tells the model what to do when the fact is absent.

Follow one sentence from text to the next token

Step through one prediction. The numbers are made up and simplified, but the order of the steps is the real order.

  1. Step 1 of 7

    1. Text

    The context so far is: Please confirm your appointment on.

Training versus inference

Compare the two activities that are easy to mix up.

QuestionTrainingInference
What happens?The model's parameters are adjusted using a large collection of textThe trained model produces tokens for a specific prompt
When?Before you use the model (and again when a new version is built)Every time you send a request
Do the parameters change?Yes, that is the purposeNo, they stay fixed
What does it need?Large amounts of data, computing power, and timeYour prompt, plus whatever the application adds to the context
What you can influenceNothing directly, as a userThe context: your instructions, sources, and examples

What happens when you send a chat message

Follow the path from your words to the paragraph on screen.

  1. Step 1 of 5

    You write a prompt

    Your text is the visible part of the context. You can include instructions, source material, and constraints.

What is the model doing?

Answer first, then reveal.

The assistant writes a three-paragraph email from your bullets.

Generation: next-token prediction in the context of your bullets and instructions. You still check names, tone, and commitments.

The assistant states a fee that was not in your message or the attached policy.

Unsupported continuation. The model filled a gap with likely-looking language. Do not send it.

The assistant uses a calculator tool and then explains the result.

The application called a tool. The arithmetic should come from the tool; the explanation is still generated language and can misstate the number.

You ask what you decided in a chat from last year that is not in this window.

That decision is not in context unless retrieved. A confident recap may be a guess.

A spam filter labels a message 'billing' without writing a reply.

That can be ordinary classification, not an LLM. Not every language-related model generates paragraphs.

Explain an LLM without treating it as a person

Explain what a large language model is to an adult beginner. Cover tokens, next-token prediction, context, and the difference between the model and the chat app. Give one example of a useful drafting task and one example of a question that needs a source. Use the fictional company Harborlight Home Services. Do not invent fees, studies, or parameter counts. Do not describe the model as a person.

Which stage is this?

Decide whether each description is tokenization, embedding, attention, or next-token generation before revealing the answer.

The sentence I need Friday off. is split into smaller pieces of text.

Tokenization. The tokenizer splits text into tokens.

Each piece is replaced by a learned list of numbers.

Embedding. The token ID is mapped to its learned vector.

While processing the word it, the model draws mostly on the earlier word heater.

Attention. The model weighs other tokens in the context.

The model scores possible continuations and the system appends Friday.

Next-token generation. Probabilities are produced and one token is chosen and appended.

Hands-on exercises

Beginner AI Beginners

Beginner: watch a fluent guess appear

Without using confidential information, try two prompts in an AI assistant if you have one. If you do not, write the prompts on paper and predict what a next-token system is being asked to do.

Prompt A: Ask a specific factual question you did not provide a source for, about the fictional company Harborlight Home Services (for example a fee, a staff count, or a founding year).

Prompt B: Paste this source text and ask the same question, with an instruction to use only the source and to say when the answer is not there:

Harborlight Home Services is a fictional local maintenance company. It schedules weekday visits. Weekend work needs manager approval. This text does not list fees or staff counts.

Write down:

  1. Whether Prompt A invented a detail.
  2. Whether Prompt B stayed inside the source.
  3. What that tells you about context.

Do not share the invented Prompt A details as if they were real.

Using only the text I provide, answer: [question]. If the text does not contain the answer, say that it is not listed. Do not invent numbers.

[paste source]

What a good result looks like: A short comparison showing that missing context invites invented continuation.

Business Business Owners and Professionals

Business: map LLM use onto a Harborlight workflow

Map one Harborlight workflow (incoming email, quoting, or post-visit follow-up) into four columns:

  • LLM may draft: language a person can edit.
  • Must come from a system of record: prices, slots, account IDs, warranties.
  • Must be a person: apologies for harm, safety, exceptions, legal wording.
  • Must not leave the company: customer personal data, unless you are on an approved tool with a policy.

Then write a one-paragraph policy for staff: when they may paste a message into an assistant, what they must remove first, and who sends the final email.

Use fictional sample lines only.

What a good result looks like: A workflow map and a short staff rule that keeps the LLM in the draft lane.

Advanced Advanced AI and Technical Development

Advanced: run a toy next-token table

Use the advanced companion code.

  1. Run the character-level model. Record the greedy continuation and the seeded sample.
  2. Explain in two sentences why greedy text can loop, and why the sample is not a reliable description of cats or mats.
  3. Add one new sentence to the tiny corpus, rerun, and note what changed.
  4. Using the tokenizer illustration, compare word splits with character splits on the same Harborlight sentence. List one practical consequence for counting context.
  5. Write a short technical paragraph connecting this toy to a real LLM: same next-token idea, different scale, still not a knowledge guarantee.

Standard library only.

What a good result looks like: Working toy output, a note on looping and sampling, and a correct scale warning.

Business

Path B: Language models in a small business

Treat the assistant as a junior writer with no company memory

Harborlight staff can get value from an LLM the same day: cleaner drafts, checklists from notes, and alternative phrasings of a notice that a manager already approved.

The assistant does not know Harborlight's real fees, the truck schedule, or which customer is on a payment plan unless that information is in the prompt or in an approved connected system.

A narrow first deployment

Allowed: draft a reply from a pasted FAQ. A person compares the draft with the FAQ and sends it.

Not allowed: paste full customer addresses, payment details, or medical notes into a consumer tool. Invent a weekend fee. Let the model mark a job complete.

Why this matches the technology

The model completes language. If the FAQ does not contain the fee, a next-token system can still emit a dollar amount that looks like a fee. Your policy has to forbid that, and your prompt has to say "if it is not in the FAQ, say so."

Measurement

Count how often staff had to correct a fact, not only how fast the first draft appeared. If corrections take longer than writing the email, the tool is not helping that template yet.

Vendor questions without the hype

When you evaluate a product, ask:

  • Where does the text go, and is that acceptable under your privacy policy?
  • Is there retrieval from your documents, or only a general model?
  • What is the context limit, in tokens, for the plan you would buy?
  • Can you turn off training on your chats if that option exists?
  • How will you keep prices and schedules in software, not in generated prose?

You do not need a research team. You need a written lane for drafts and a closed door for records.

Common misconceptions

Myth The model looks up a stored answer for every question.
Reality

Generation is usually next-token prediction in context. Some products add retrieval or tools, but the default is not a guaranteed lookup.

Myth If the answer is grammatical, the facts are probably right.
Reality

Fluency is what the training objective encourages. Facts still need a source and a check.

Myth The assistant remembers your business the way a colleague would.
Reality

It only uses trained parameters, the current context window, and whatever the application retrieves or stores. Last month's decision is not automatically available.

Myth Every AI product that handles text is a large language model.
Reality

Classifiers, search, and rules-based templates can handle text without generating long passages. Ask what the system actually does.

Myth You can trust a citation the model typed because it looks like a link or a paper title.
Reality

Titles and URLs can be assembled as likely tokens. Open the source yourself.

Knowledge check

Answer every question, then check your answers. You need 70% to pass. A question marked "select all that apply" counts as correct only when you choose every correct answer and no others.

Question 1 Which statement best describes a large language model in this lesson?
Question 2 Which statements about tokens are accurate here? Select all that apply. (select all that apply)
Question 3 When a chat assistant writes a long paragraph, it is usually selecting a complete pre-written essay that already contained your name.
Question 4 What is the context window, as used in this lesson?
Question 5 You ask an assistant for Harborlight's weekend fee and do not provide a policy. The reply contains a dollar amount. What happened?
Question 6 Which layers are distinguished in this lesson? Select all that apply. (select all that apply)
Question 7 A grammatical, confident sentence from a language model is sufficient proof that the sentence is true.
Question 8 Which Harborlight task is the best first use of an LLM according to this lesson?
Question 9 Why do prompts matter for language models? Select all that apply. (select all that apply)
Question 10 What should you do when an answer depends on a business fact?
Question 11 In this lesson, what is an embedding?
Question 12 Which statements about attention are accurate? Select all that apply. (select all that apply)
Question 13 When you chat with an assistant, your messages update the model's trained parameters as you type.

Advanced Advanced AI and Technical Development

Advanced companion

This companion connects Lesson 3's fitted network fθf_\theta to language. A typical language model defines a distribution over a discrete vocabulary of tokens. Generation is sequential sampling (or greedy choice) from that distribution. The Python toys use the standard library only. They operate on characters, not a production subword vocabulary, and they are far too small to resemble a product LLM.

The teaching point is the objective, not the architecture: likely continuation is not the same as a query against a verified knowledge base.

1. Tokens and a categorical output

Let the vocabulary be a finite set V\mathcal{V}. A tokenizer maps a string to a sequence t1,t2,…,tnt_1, t_2, \ldots, t_n with each tk∈Vt_k \in \mathcal{V}. Subword tokenizers keep ∣V∣|\mathcal{V}| manageable while still representing rare words as several tokens. Character-level models, used in the code example, are the extreme where V\mathcal{V} is the character set.

A language model supplies, at each position,

pθ(tk∣t1,…,tk−1)p_\theta(t_k \mid t_1, \ldots, t_{k-1})

often by computing logits z∈R∣V∣z \in \mathbb{R}^{|\mathcal{V}|} and a softmax

pi=ezi/T∑jezj/Tp_i = \frac{e^{z_i / T}}{\sum_j e^{z_j / T}}

where T>0T > 0 is a temperature. T→0T \to 0 approaches greedy choice of the largest logit. Larger TT flattens the distribution. Product defaults vary; do not treat a toy temperature as a vendor setting.

The context window is a bound on how many tokens of (t1,…,tk−1)(t_1, \ldots, t_{k-1}) (and, in some systems, of the tokens being produced) participate in that conditioning.

2. Training as next-token loss

Given a corpus tokenized as sequences, a standard pretraining loss is average negative log-likelihood:

L(θ)=−1N∑klog⁡pθ(tk∣t<k)L(\theta) = - \frac{1}{N} \sum_{k} \log p_\theta(t_k \mid t_{<k})

This is supervised learning where the "label" is the next token already present in the text. The model is not trained, in this objective, to emit "I do not know" when a fact is absent. It is trained to assign high probability to continuations that occurred in training data.

Instruction tuning and preference training add later datasets in which the targets are responses to prompts, or comparisons between responses. Those stages can change typical behavior. They still do not install a guaranteed fact table.

A deployed assistant is fθf_\theta plus decoding code plus application policy. Evaluating only L(θ)L(\theta) on generic text does not measure factuality on Harborlight's FAQ.

3. Decoding is a policy

Even with pθp_\theta fixed, the next token is chosen by a decoding rule:

  • Greedy: tk=arg max⁡ipθ(i∣t<k)t_k = \operatorname*{arg\,max}_i p_\theta(i \mid t_{<k}).
  • Sampling: draw from pθp_\theta, sometimes after restricting to a top-kk or nucleus (top-pp) set.
  • Beam search: keep several partial sequences (more common in older translation systems than in chat decoding).

Greedy decoding on a tiny character model often loops, because a frequent cycle in the count table becomes self-reinforcing. Sampling breaks the loop and can look more "creative" while remaining ungrounded.

Temperature, top-kk, and nucleus sampling are decoding knobs. They are not truth knobs. Lowering temperature can make outputs more repetitive; it does not certify facts.

When an application adds retrieval, the retrieved text is concatenated (or otherwise fused) into the conditioning context. The model can still ignore it. Grounding is a system property, not an automatic result of having a search API.

4. Why likelihood is not grounding

Let ss be a fluent sentence that happens to be false, and s′s' a true but awkward sentence. Nothing in next-token loss requires pθ(s)<pθ(s′)p_\theta(s) < p_\theta(s'). Training data contains errors, fiction, and confident prose. The loss prefers typical language.

Hallucination, in this course's practical sense, is produced content that is not supported by the allowed source. It is expected whenever the user asks for a closed-world fact that is missing from context and the decoder is still required to emit tokens.

Mitigations are system design: put sources in context, instruct the model to abstain, validate structure with ordinary software, call a tool for arithmetic, and keep a person on high-impact sends. None of those is provided by stacking more layers alone.

5. Embeddings and attention, more precisely

Let the vocabulary have VV tokens and let the model width be dd. An embedding matrix EE holds one learned row of length dd per token. The input for the token at position kk combines its embedding with position information:

xk=E[tk]+pkx_k = E[t_k] + p_k

Stack the inputs for the context as the rows of a matrix XX. Each attention head learns three projections, producing queries, keys, and values:

Q=XWQ,K=XWK,V=XWVQ = X W_Q, \quad K = X W_K, \quad V = X W_V

Scaled dot-product attention then mixes the values according to how well each query matches each key:

Attention(Q,K,V)=softmax(QK⊤dk)V\mathrm{Attention}(Q, K, V) = \mathrm{softmax}\left(\frac{Q K^{\top}}{\sqrt{d_k}}\right) V

Here dkd_k is the key dimension, and the softmax is applied row by row so each token's weights sum to 1. Dividing by dk\sqrt{d_k} keeps the scores in a range where the softmax does not become extremely peaked.

In a language model used for generation, a causal mask sets the scores for later positions to minus infinity before the softmax, so position kk can attend only to positions up to kk. That is the technical form of "the model attends only to tokens already in the context."

A transformer layer runs several heads in parallel, combines their outputs, and passes the result through a small feed-forward network, with residual connections and normalization around both parts. Many such layers are stacked. The final representation is projected to one score per vocabulary token, and a softmax turns those scores into the next-token distribution from section 1.

Training and inference in these terms. During training, EE, the projection matrices, and every other parameter are updated by gradient descent on the loss in section 2. During inference they are fixed: the same calculation simply runs forward, one new token at a time. Implementations commonly cache the keys and values of earlier tokens so each new step does not recompute them.

6. From toy counts to transformers

The code example stores p(b∣a)p(b \mid a) as character bigram counts. A transformer language model conditions on a long context with attention, so p(tk∣t<k)p(t_k \mid t_{<k}) can depend on distant tokens, not only the previous character. The objective is still next-token prediction.

You do not need to implement attention to use an LLM well. You do need to remember that scale and architecture change fluency and range, not the logical gap between "likely token" and "verified record."

Lesson 5 will treat the prompt as an engineered prefix t1,…,tmt_1, \ldots, t_{m} that you control. The mathematics here is why that prefix matters: every generated token is conditioned on it.

A character bigram 'language model'

# Educational example only. Standard library only.
# Character-level bigrams estimated on a tiny fictional corpus.
# This is not a neural network and not a product LLM.

from collections import Counter, defaultdict
import random

corpus = (
    "the cat sat on the mat. the cat sat. "
    "the man can map the path. a cat can nap."
)

pairs = defaultdict(Counter)
for a, b in zip(corpus, corpus[1:]):
    pairs[a][b] += 1


def greedy_continue(start, length=60):
    ch = start
    out = [ch]
    for _ in range(length):
        if ch not in pairs:
            break
        ch = pairs[ch].most_common(1)[0][0]
        out.append(ch)
    return "".join(out)


def sample_continue(start, length=60, seed=0):
    random.seed(seed)
    ch = start
    out = [ch]
    for _ in range(length):
        if ch not in pairs:
            break
        letters, counts = zip(*pairs[ch].items())
        ch = random.choices(letters, weights=counts, k=1)[0]
        out.append(ch)
    return "".join(out)


print("corpus length (characters):", len(corpus))
print("greedy from 't':", greedy_continue("t"))
print("sample seed 0 from 't':", sample_continue("t", seed=0))
print("This toy can loop or babble. It does not know anything about cats.")

Counts of 'which character followed which' already show next-token thinking. Greedy decoding often loops on this corpus. Sampling produces different nonsense. Neither result is knowledge about the world. A real LLM uses a huge neural network and a subword vocabulary; the objective is still likely continuation.

Word pieces versus characters, on one sentence

# Educational example only. Standard library only.
# A production tokenizer is not whitespace splitting.
# This contrast is enough to show that 'length' depends on the units.

sentence = "Harborlight: please reschedule Friday's visit."

wordish = sentence.split()
chars = list(sentence)

print("whitespace units:", len(wordish), wordish)
print("characters:", len(chars))
print(
    "A real tokenizer might split 'reschedule' or keep 'Harborlight' "
    "as one or more subword tokens. Context limits count those tokens, "
    "not English words and not pages."
)

Whitespace splitting is not a neural tokenizer. The printout is still enough to see that the same sentence has different lengths in different unit systems. Product context windows are token budgets.

Experiment: Change decoding, watch the toy text change

Run the bigram example. Copy the greedy string and the seed-0 sample. Then run sample_continue with seed=1. Explain in three sentences: what stayed similar, why greedy looped (if it did), and why none of the strings is a source for a customer email.

Experiment: Count units on a long paste

Paste a public FAQ or a fictional one-page policy into a small script (or count by hand on a short paragraph). Compare word count, character count, and a rough token estimate such as words times 1.3. State why a context window could fill before you 'feel' the document is long. Do not claim your estimate is the vendor's tokenizer.

Experiment: Design an abstain rule

Write a decoding-time policy in plain language for Harborlight: when the model must say 'not in the FAQ', how software will reject a reply that contains a dollar amount if the FAQ has none, and who sends the email. Keep the LLM in the draft role.

Experiment: Compute one attention step by hand

Use one query q=(1,0)q = (1, 0) and three tokens with keys k1=(1,0)k_1 = (1, 0), k2=(0,1)k_2 = (0, 1), k3=(1,1)k_3 = (1, 1) and values v1=(2,0)v_1 = (2, 0), v2=(0,2)v_2 = (0, 2), v3=(1,1)v_3 = (1, 1). Skip the dk\sqrt{d_k} scaling for this exercise. (1) Compute the dot product of qq with each key. (2) Apply the softmax to those three scores. (3) Multiply each value by its weight and add them up. You should get weights of about 0.42, 0.16, and 0.42, and an output of about (1.27, 0.73). Then explain in one sentence why tokens 1 and 3 influence the output more than token 2.

Resources

Completion checklist

How was this lesson?