AI curriculum

Course outline for AI Infrastructure

An inference service can answer individual requests yet fail under load or route sensitive data incorrectly. Design a bounded gateway policy that separates resource needs, admission decisions, and fallback quality. Compare supplied memory profiles for quantization and context growth, and include input, output, and cache charges in the cost worksheet. Use synthetic load rather than treating a successful API call as a capacity guarantee.

Technology: Cloud and Platform Engineering
Duration: 6 hours
Modules: 3 · 2 hours each
Level: Level 3

About AI Infrastructure

AI Infrastructure Course Objectives

Estimate inference resource needs from supplied workload, quantization, and context-memory traces.
Separate model, provider, and application gateway responsibilities.
Implement bounded admission and routing rules in a mock gateway.
Compare queue wait, service latency, quality, and operating cost, including supplied input/output/cache accounting.
Reject a fallback that violates data-handling or quality constraints.

Pre-requisites

Familiarity with cloud deployment, APIs, and basic programming.
Understanding of inference, tokens, context resources, and model sampling, equivalent to How Large Language Models Work.
Understanding of compute, networking, identity, and persistence responsibilities, equivalent to Cloud Foundations for AI Engineers.

Before this course

Complete the required courses or bring equivalent experience. Recommended courses add useful depth; they are not enrolment gates.

Required preparation

Recommended preparation

Understands inference resource behavior and cloud responsibilities before serving/routing models; local inference and provider operations deepen comparisons.

Find your place in the learning map

Lab Setup

Programming runtime, local mock endpoints, and supplied synthetic load generator.
Dated official-reference profiles for Bedrock, Microsoft Foundry, Google Cloud managed AI, and Cloudflare Workers AI/AI Gateway with API, quota, cost, and data-handling fixtures.
No GPU, paid account, or live provider is required. Optional local inference uses nonconfidential prompts; local execution alone does not establish privacy.

Detailed Course Outline

Module 1: Workload and serving resources

Model serving: Request handling, model loading, batching, and response delivery; distinguish a working endpoint from measured throughput and reliable service behavior.
GPUs: Weight storage, runtime memory, and context growth; compare supplied quantization profiles without assuming reduced memory preserves task quality.
Inference infrastructure: Compute, network paths, queueing, and recovery responsibilities; estimate workload capacity from supplied service traces rather than individual successful requests.

Practical outcome: Produce a workload/resource worksheet distinguishing weight size, runtime memory, context growth, and measured service time; identify untested capacity assumptions.

Module 2: Gateway decisions under load

AI gateways: Application-level identity, request limits, accounting, and provider boundaries; locate policy enforcement separately from model generation and provider infrastructure.
Model routing: Select endpoints by task quality, latency, availability, and permitted data scope; record why a route meets the original task contract.
Admission control and fallback constraints: Bound queue depth, reject excess work explicitly, and evaluate fallback quality and data handling before redirecting requests during overload.

Practical outcome: Implement mock per-stage routing with an explicit queue limit, overload response, and minimum fallback quality and data-handling rules.

Module 3: Managed AI platforms and operating cost

Local vs cloud inference: Compare hardware capacity, service latency, network exposure, logs, and operating ownership; local execution alone does not establish privacy.
AWS Bedrock: Compare model access, quotas, invocation identity, and cross-region data paths; distinguish managed foundation-model APIs from SageMaker AI model deployment and application-owned gateway policy.
Managed model platforms: Compare Microsoft Foundry, Google Cloud managed AI (Vertex AI lineage), and Cloudflare Workers AI/AI Gateway using dated profiles; separate model access, agent hosting, retrieval, quotas, and accounting.

Practical outcome: Compare dated profiles for Amazon Bedrock, Microsoft Foundry, Google Cloud managed AI, and Cloudflare Workers AI/AI Gateway against local inference; distinguish model access, agent hosting, retrieval, API/region support, quotas, and data responsibilities.

Practical exercise

Submit a mock gateway policy, workload worksheet, provider responsibility matrix, and load-test report for two endpoint fixtures.
Run normal, overloaded, unavailable-provider, and unsuitable-fallback cases; record queue wait separately from service latency.
Accept every valid request within the fixture budget or return the documented rejection; keep queue depth at or below its configured limit in all cases.
Reject supplied fallbacks that breach quality, API compatibility, or data-scope rules; compare provider token/cache charges and quota assumptions without treating managed platforms as interchangeable.

What you will produce

Test a workload-driven inference gateway with bounded overload and fallback behavior.

How the work is checked

Queue depth never exceeds the declared limit during supplied overload tests.
Every route or rejection records its quality and data-handling constraints.
The worksheet compares supplied quantization/context memory profiles and reconciles input, output, and cache charges without assuming unchanged quality.
The comparison separates wait time, service time, total operating cost, and unverified capacity assumptions.

Six-hour mock gateway lab with a bounded cross-cloud comparison, not four provider implementations. Record product/API dates and current regional/preview limits. Microsoft Foundry was previously Azure AI Foundry; Google Cloud documentation includes Vertex AI lineage under Gemini Enterprise Agent Platform. Live integration, GPU serving, and autoscaling require separate preparation.

When another approach is better

A single low-volume endpoint may not justify a gateway and routing policy.
Local inference is unsuitable when measured hardware capacity or task quality misses the workload requirements.
This mock load test cannot replace GPU benchmarking or managed-service capacity validation.

How we train

Ask about exercises, instructor feedback and the prior knowledge you need. Tool-specific courses marked provisional may become modules in a broader course.

Discuss your learning goals