AI curriculum

Course outline for How Large Language Models Work

A model may lose useful context or produce different answers even when a request looks similar. Trace how text becomes tokens and how generation uses context, then inspect resource and sampling trade-offs. Distinguish model weights, request context, and external memory before deciding what to include or retrieve.

Technology: AI Foundations
Duration: 6 hours
Modules: 3 · 2 hours each
Level: Level 2

About How Large Language Models Work

How Large Language Models Work Course Objectives

Trace text through tokenization, embeddings, and a guided attention example.
Distinguish model weights, request context, and persistent application memory.
Interpret an inference trace with input, output, and reported cache usage.
Explain the difference between KV cache and prompt caching.
Compare repeated outputs under documented sampling conditions without assuming determinism.
Select a bounded context budget and state limits of reasoning-style and multimodal outputs.

Pre-requisites

Artificial Intelligence Foundations or equivalent understanding of generative AI and training versus inference.
Comfort reading simple numerical examples.
A notebook is optional; programming and specialized hardware are not required.

Before this course

Complete the required courses or bring equivalent experience. Recommended courses add useful depth; they are not enrolment gates.

Required preparation

Artificial Intelligence Foundations

Understands generative AI and training versus inference before studying tokens, sampling, and context resources.

Find your place in the learning map

Lab Setup

Computer with a browser and a spreadsheet or notebook.
Supplied tokenizer examples, small attention diagrams, version-labeled inference traces, and synthetic text/image fixtures.
A small local model is optional; no GPU, paid API, or universal provider cache behavior is assumed.

Detailed Course Outline

Module 1: Represent a request (2 hours)

Tokenization: Text segments, token identifiers, and sequence length; relate the supplied token trace to input representation and request budgeting.
Embeddings: Learned vector representations of tokens and semantic relationships; distinguish numerical similarity from factual truth or persistent application memory.
Transformer architecture: Token representations, attention blocks, and next-token prediction; trace a guided request without requiring a complete transformer implementation.
Attention: Queries, keys, values, and weighted combinations in a small numerical example; explain how token representations incorporate available context.

Practical outcome: Annotate a small token and attention example, explaining representation and next-token prediction without confusing embedding similarity with factual truth.

Module 2: Budget context and inference resources (2 hours)

Context windows: Request messages, token limits, and evidence selection; distinguish supplied context from model weights and external persistent state.
Inference: Request processing, next-token generation, and usage traces; reconcile reported input and output categories without assuming universal accounting rules.
KV cache: Reused attention keys and values during generation; distinguish runtime computation reuse from provider prompt caching and durable memory.

Practical outcome: Inspect message and usage traces, separate weights, request context, and external memory, and choose supplied versus retrieved information while distinguishing KV cache from provider prompt caching.

Module 3: Interpret generation behavior (2 hours)

Sampling: Next-token distributions, temperature, and top-p settings where supported; compare repeated outputs without treating low temperature as guaranteed determinism.
Reasoning models: Reasoning-style generation, reported resource usage, and visible summaries; distinguish additional computation from formal proof or guaranteed answer correctness.
Multimodal models: Text and image inputs, representation limits, and evidence availability; interpret supplied examples without assuming reliable perception or complete recall.

Practical outcome: Compare repeated recorded generations, documented sampling settings, and text/image input limitations; account for reported usage without treating visible reasoning as proof or large context as guaranteed recall.

Practical exercise

Produce an annotated inference trace, context-budget worksheet, and repeated-output comparison.
Retain the evidence needed for a valid question and flag an answer unsupported by the available context.
Reconcile input, output, and separately reported cache categories for three supplied runs using the fixture's accounting rules.
Compare at least three repetitions per setting and explain what the observed variation does and does not establish.

What you will produce

Explain a bounded inference trace and make a context-budget decision using generation behavior and reported resource evidence.

How the work is checked

The trace separates token representation, generation, and external application state.
KV cache and prompt caching are described as different mechanisms.
Usage totals reconcile under the supplied accounting conventions.
The output comparison avoids determinism, recall, and reasoning-proof claims.

Six hours of guided internals and recorded inference analysis, with a small numerical attention example rather than a transformer implementation. Budget approximately one hour per module for the worksheet or trace exercise. Training and serving-capacity engineering are advanced extensions.

When another approach is better

Do not solve missing-evidence problems by increasing the context window without testing retrieval and answer support.
Do not treat low temperature or a reasoning-style response as a correctness guarantee.

How we train

Ask about exercises, instructor feedback and the prior knowledge you need. Tool-specific courses marked provisional may become modules in a broader course.

Discuss your learning goals