AI curriculum

Course outline for Local and Small Language Models

Hosted inference may not fit a workload's data or operating constraints, but a local model may exceed the available machine or fail the task. Compare two small-model configurations using the same inputs and record quality, latency, memory, and operating effort. Choose local or remote inference with explicit licensing and privacy assumptions.

Technology: AI Foundations
Duration: 4 hours
Modules: 2 · 2 hours each
Level: Level 2

About Local and Small Language Models

Local and Small Language Models Course Objectives

Select a small-model configuration against a stated workload and memory limit.
Distinguish quantization, model format, and runtime compatibility.
Run or inspect a repeatable local inference workflow.
Compare observed quality, latency, peak memory, and review effort.
Check supplied licensing, data-handling, and logging constraints.
Justify a local or remote choice including remaining security responsibilities.

Pre-requisites

Artificial Intelligence Foundations or equivalent understanding of model inputs, outputs, and limitations.
Basic command-line skills.
Familiarity with language model inputs and outputs; inference internals are helpful but not required.

Before this course

Complete the required courses or bring equivalent experience. Recommended courses add useful depth; they are not enrolment gates.

Required preparation

Artificial Intelligence Foundations

Recommended preparation

How Large Language Models Work

Understands model inputs and outputs and has basic shell skills; LLM internals help explain memory and runtime trade-offs.

Find your place in the learning map

Lab Setup

Computer with a terminal and a prepared compatible local runtime; use a model selected for available memory and verified license terms.
Synthetic tasks, version-labeled manifests, and benchmark traces provide a complete alternative when local inference is impractical.
GPU and remote accounts are optional; bind any local endpoint to a restricted interface and use no confidential prompts.

Detailed Course Outline

Module 1: Match a model to a constrained machine (2 hours)

Small language models: Task fit, parameter scale, and constrained resource budgets; compare observed quality and review effort rather than assuming smaller models suffice.
Local inference: Model loading, runtime configuration, and repeatable requests; distinguish cold starts from warm runs and local operation from remote API access.
Quantization: Reduced numerical precision, weight storage, and runtime compatibility; compare measured quality and memory without assuming a universal accuracy penalty.
Model formats: Weight packaging, versioned manifests, and runtime support; check GGUF compatibility where applicable rather than treating formats as interchangeable.

Practical outcome: Inspect version-labeled model and runtime manifests, choose a compatible format such as GGUF where supported, and compare supplied quantization configurations without assuming a universal quality penalty.

Module 2: Measure deployment trade-offs (2 hours)

Hardware considerations: Weight memory, inference overhead, and available compute; distinguish total measured memory from file size and record unavailable measurements explicitly.
Ollama / llama.cpp-style ecosystems: Runtime wrappers, supported formats, and endpoint configuration in supplied manifests; compare access paths without assuming identical capabilities across versions.
Privacy and deployment trade-offs: Data exclusions, licensing, logs, network exposure, and operating responsibility; compare local and remote choices without equating locality with security.

Practical outcome: Run or inspect the same task through two local configurations, distinguish weight size from total inference memory, and compare licensing, exposed endpoints, logging, operating effort, and remote alternatives.

Practical exercise

Produce a two-configuration benchmark worksheet and a local-versus-remote decision record.
Accept a compatible configuration that meets the stated task criteria; reject an incompatible format or a configuration that exceeds the supplied memory budget.
Compare at least eight task fixtures and three timed repetitions per configuration, recording quality, latency, measured memory where available, and review effort.
Identify data that must be excluded and residual privacy risks from logs, dependencies, and network exposure; do not assume local execution is secure by itself.

What you will produce

Select a local or remote inference path using a repeatable small-model comparison and explicit resource and data constraints.

How the work is checked

Configuration compatibility and licensing assumptions are recorded.
The comparison uses identical task fixtures and separates quality from speed.
Resource conclusions use measured or supplied runtime evidence rather than weight size alone.
The decision includes maintenance, review effort, and residual data-handling risks.

Four hours using preloaded models or recorded traces; installation and downloads are instructor preparation. The eight tasks and timed repeats use a supplied runner, not repeated manual setup. Distinguish measured memory from unavailable measurements and cold-start from warm-run latency. Exercises fit within module time; training and production serving are outside scope.

When another approach is better

Do not choose local inference when available memory or observed task quality cannot meet the requirement.
Do not self-host solely for privacy when maintenance, endpoint access, logging, and dependency risks cannot be controlled.

How we train

Ask about exercises, instructor feedback and the prior knowledge you need. Tool-specific courses marked provisional may become modules in a broader course.

Discuss your learning goals