AI curriculum

Course outline for AI Security Foundations

Untrusted retrieved text can try to turn an AI suggestion into a privileged tool action. Trace that path in a small mock application and define controls at input, data, and mutation boundaries. Test prevention, detection, and response evidence with harmless fixtures rather than relying on a defensive prompt alone.

Technology: AI Security, Governance and Trust
Duration: 6 hours
Modules: 3 · 2 hours each
Level: Level 2

About AI Security Foundations

AI Security Foundations Course Objectives

Map assets, untrusted inputs, trust boundaries, and protected-state invariants.
Distinguish prompt injection from direct attempts to bypass model safeguards.
Test allowed reads and blocked data exposure or writes using harmless fixtures.
Review tool authority and dependency risks using supplied security guidance.
Link each control to prevention, detection, and response evidence.

Pre-requisites

AI capability and limitation awareness equivalent to Artificial Intelligence Foundations.
Basic application security and data-access concepts.
Ability to inspect a request and its possible effects; coding is optional with the supplied guided mock interface.

Before this course

Complete the required courses or bring equivalent experience. Recommended courses add useful depth; they are not enrolment gates.

Required preparation

Artificial Intelligence Foundations

Recommended preparation

Understands AI behavior and has basic application-security/data-access knowledge; application practice helps trace untrusted inputs to actions.

Find your place in the learning map

Lab Setup

Computer with a browser and a disposable local mock application or supplied guided execution traces.
Use synthetic state, dummy secrets, harmless instructions, and a dated official OWASP guidance excerpt; no production tokens or public targets.
Keep authentication, authorization, and resource enforcement outside model prompts; coding is optional for the guided analysis path.

Detailed Course Outline

Module 1: Untrusted input and protected assets

AI threat models: Map protected assets, attackers, untrusted inputs, and trust boundaries; define state invariants along the path from content to tool execution.
Prompt injection: Trace instructions embedded in retrieved or external content; prevent untrusted text from changing tool authority or overriding application policy.
Jailbreaking: Distinguish direct attempts to bypass model safeguards from injected external instructions; compare harmless fixtures without treating model refusal as complete protection.

Practical outcome: Draw an untrusted-content-to-tool path and define a safety invariant that model-generated instructions cannot override.

Module 2: Data and execution exposure

Data leakage: Identify disclosure paths through outputs, context, and logs; test dummy-secret exclusion and minimize data accessible to model interactions.
Tool abuse: Constrain tool operations, parameters, and protected targets outside prompts; verify forbidden mutations fail while legitimate reads remain available.
Model abuse: Examine misuse of generation and excessive invocation; compare resource limits, allowed-use boundaries, and detection signals in harmless supplied cases.

Practical outcome: Test dummy-secret protection, limited tool authority, and resource caps; record why an allowed answer does not authorize a write.

Module 3: Dependencies and defensive evidence

Supply-chain risk: Inspect supplied dependency and tool-component risks; identify trusted sources, update ownership, and exposure introduced by compromised or unexpected components.
OWASP guidance: Map a dated official-guidance excerpt to concrete threats and controls; distinguish useful review coverage from comprehensive assurance or claimed conformance.
Control verification and response: Separate prevention tests, detection records, and response actions; confirm unchanged protected state and document attack classes the fixtures do not test.

Practical outcome: Produce a threat/control matrix using a supplied dated official-guidance excerpt and distinguish blocked attempts, detection records, and response actions.

Practical exercise

Submit a trust-boundary diagram and threat/control matrix with a harmless fixture test report.
Compare an allowed read with injected instructions requesting a write, dummy-secret disclosure, or excessive tool use.
Require every seeded forbidden mutation to leave protected synthetic state unchanged and verify the valid read still succeeds.
Confirm no dummy secret appears in output or logs, and document detection and response evidence plus at least one untested attack class.

What you will produce

Verify a layered control against an untrusted-input-to-tool threat in an isolated mock application.

How the work is checked

The diagram names protected assets and explicit trust boundaries.
All seeded forbidden mutations are denied with protected state unchanged and valid reads still work.
The report separates prevention, detection, response, and untested threats.

Six-hour defensive mock lab; current guidance is a review input, not a promise of comprehensive security or OWASP conformance.

When another approach is better

A prompt-only guardrail is insufficient for privileged tools or sensitive data.
This isolated fixture test cannot replace a full penetration test or dependency security review.
Use simpler restricted access when the task does not need tool execution.

How we train

Ask about exercises, instructor feedback and the prior knowledge you need. Tool-specific courses marked provisional may become modules in a broader course.

Discuss your learning goals