AI curriculum

Course outline for Formal Methods for Reliable AI Systems

An agent must not publish a model-generated report using approval granted to an earlier draft. Specify the deterministic approval controller and explore a small model for stale approval, retries, and cancellation. Compare bounded model checking with a guided proof obligation without claiming to prove the model's answers correct.

Technology: Software Engineering Foundations
Duration: 4 hours
Modules: 2 · 2 hours each
Level: Level 2

About Formal Methods for Reliable AI Systems

Formal Methods for Reliable AI Systems Course Objectives

Specify an agent's approval controller with explicit states and assumptions.
Bind tool execution to approval of the exact current draft.
Express a bounded action-budget invariant and distinguish safety from progress.
Interpret stale-approval, duplicate-request, and cancellation counterexamples.
Correct a supplied bounded TLA+ model and inspect a Lean proof obligation.
Distinguish verified controller properties from model answer quality and implementation correctness.

Pre-requisites

Software Engineering Foundations for AI Engineers or equivalent ability to describe software boundaries and test failure cases.
Programming experience and basic logical reasoning.
Familiarity with concurrent or distributed system behavior.

Before this course

Complete the required courses or bring equivalent experience. Recommended courses add useful depth; they are not enrolment gates.

Required preparation

Software Engineering Foundations for AI Engineers

Recommended preparation

Can describe software behavior and failure cases before expressing invariants; language semantics and recovery scenarios deepen the models.

Find your place in the learning map

Lab Setup

Computer with a code editor and a prepared, version-pinned TLA+ checker environment.
Supplied small models and checker outputs; a prepared Lean environment or annotated proof fixture supports guided comparison.
All exploration is local and finite; installation troubleshooting and full theorem-prover development are outside the lab.

Detailed Course Outline

Module 1: Specify an agent's approval boundary (2 hours)

Formal specification: Specify an agent controller's draft versions, authority, action budget, and commit assumptions; model generated proposals as inputs rather than trusted decisions.
State machines: Model draft, approval, execution, cancellation, and completion states; distinguish canceling pending work from reversing an already committed tool effect.
Invariants: Require exact-version approval and bounded accepted actions; distinguish enforced controller safety from claims about generated content being true or useful.
Model checking: Finite state exploration, stated bounds, and counterexample traces; distinguish a successful bounded check from universal implementation correctness.

Practical outcome: Model generated-draft versions, approval, cancellation, and publication; define exact-version approval and action-budget invariants and inspect a stale-approval counterexample.

Module 2: Compare formal evidence (2 hours)

TLA+: State variables, transition actions, and invariant checks in a supplied model; correct stale approval and inspect the checker results.
Lean: Proof obligations, assumptions, and checked logical steps in an annotated example; compare theorem evidence with bounded behavioral exploration.
Formal reasoning for distributed systems: Inspect concurrent draft edits, retries, and remote-effect assumptions; separate bounded controller evidence from fairness, implementation correctness, and exactly-once execution claims.

Practical outcome: Correct one supplied bounded TLA+ controller model, inspect a Lean proof obligation, and trace duplicate requests and cancellation without assuming remote effects are automatically reversed.

Practical exercise

Produce a corrected agent-controller model, invariant record, failing trace, and assumption list using synthetic drafts and mock publication actions.
Permit approved-current-version publication; block stale approval, a duplicate accepted action, and execution beyond the supplied action budget.
Run the finite model under documented bounds and retain before/after checker evidence; inspect cancellation before and after the modeled commit point.
State a progress condition with fairness assumptions and distinguish controller guarantees from model truthfulness, ontology consistency, and production implementation evidence.

What you will produce

Check a bounded agent approval controller and explain version, action-budget, retry, and cancellation assumptions without claiming to verify model answers.

How the work is checked

The invariant requires approval of the exact published version.
The corrected bounded model blocks stale approval, duplicate accepted actions, and action-budget violations under its stated assumptions.
Safety, progress, and fairness assumptions are stated separately.
The comparison distinguishes controller properties, proof obligations, model answer quality, and implementation evidence.

Four hours amending one prepared finite agent-controller model and inspecting one Lean proof fixture. Action-budget and cancellation cases use supplied transitions. Progress needs explicit fairness assumptions. Checked controller properties neither prove generated answers correct nor establish exactly-once external effects; no live agent or production protocol proof is included.

When another approach is better

Do not invest in formal tooling before the requirement and modeled system boundary are clear.
Do not use a bounded successful check as a replacement for implementation tests or as a universal distributed-system proof.

How we train

Ask about exercises, instructor feedback and the prior knowledge you need. Tool-specific courses marked provisional may become modules in a broader course.

Discuss your learning goals