AI curriculum
Course outline for Local and Small Language Models
Run a language model on a machine you operate rather than relying on a hosted model API. Choose a model that fits the task and hardware, load it with Ollama, and call it from a terminal and a small client. Compare configurations and understand quantization, formats, memory, licensing, and data-handling trade-offs.
Local and Small Language Models Course Objectives
- Choose compatible model files and a runtime for available hardware.
- Set up and run local inference and make a programmatic request.
- Explain quantization, model formats, and inference-memory requirements.
- Compare quality and measured performance under controlled conditions.
- Assess licensing, data paths, network exposure, and maintenance responsibilities.
Pre-requisites
- Artificial Intelligence Foundations or equivalent model-input/output and limitation knowledge.
- Basic terminal commands and the ability to edit a small configuration file.
- How Large Language Models Work helps explain inference memory; Applied Generative AI and a GPU are not mandatory prerequisites.
Before this course
Complete the required courses or bring equivalent experience. Recommended courses add useful depth; they are not enrolment gates.
Required preparation
Recommended preparation
Readiness checks
- Artificial Intelligence Foundations or equivalent model-input/output and limitation knowledge.
- Basic terminal commands and the ability to edit a small configuration file.
- How Large Language Models Work helps explain inference memory; Applied Generative AI and a GPU are not mandatory prerequisites.
Understands model inputs, outputs, and limitations and can use basic terminal commands and edit a small configuration file; Applied Generative AI and a GPU are not mandatory prerequisites.
Find your place in the learning mapLab Setup
Delivery requirements: the following environment, materials, access, and exercise timing must be prepared and verified before delivery. This outline does not certify that a lab package is ready.
A preflight-checked learner or allocated lab machine with enough memory and storage for the selected small model. Stage large downloads before class; guided time still includes runtime configuration and actual operation. Exact requirements depend on the verified model and precision. A GPU is optional only if the CPU path meets the lab's timing needs. Recorded traces are not an equivalent substitute for local execution.
Detailed Course Outline
Module 1: Models, runtimes, and a first local run (2 hours)
- Small model variants: Small models, parameter count, base and instruction-tuned variants.
- Model identity and formats: Model identity, licenses, tokenizer dependencies, and supported formats.
- Ollama setup and first inference: Ollama setup, model acquisition, loading, and a first request.
- Changed-input reruns: A changed-input rerun and inspection of the actual response.
- llama.cpp and GGUF comparison: llama.cpp roles and GGUF examples, compared with the selected runtime.
Practical outcome: Run a compatible small model and record its identity and runtime settings.
Module 2: Hardware, quantization, and local integration (2 hours)
- Inference memory and hardware: RAM, VRAM, CPU/GPU execution, and total inference memory.
- Quantization trade-offs: Compare weight precision, quantization, storage needs, and observed task-quality trade-offs.
- Context and generation limits: Observe how context and generation length affect memory requirements and performance.
- Local HTTP clients: Local HTTP requests and a small programmatic client.
- Runtime operation: Operate the runtime lifecycle, endpoint binding, logging, and cleanup for local inference.
Practical outcome: Invoke the local endpoint and observe how a configuration change affects behavior.
Module 3: Measuring fit and choosing an execution path (2 hours)
- Comparable tasks and settings: Fixed tasks, output checks, and comparable generation settings.
- Cold and warm measurements: Cold starts, warm runs, repeated timings, and output throughput.
- Measured quality and resources: Compare quality, latency, and actual memory measurements where the runtime supports them.
- Local and hosted trade-offs: Local versus hosted inference, licensing, updates, and operating effort.
- Local privacy limits: Privacy limits from logs, network access, dependencies, and retained data.
Practical outcome: Compare two configurations and justify a local or hosted choice.
Practical exercise
- Load a small instruction-tuned model and issue a request through the runtime and a client script. Change an input or supported setting and rerun. Compare two compatible configurations on the same small task set. Use two precisions of the same model where available to isolate quantization effects; otherwise state which differences prevent that causal interpretation. Record unavailable memory measurements explicitly rather than inventing values.
What you will produce
Operate an Ollama model runtime, call its local endpoint, and compare actual quality and performance under controlled settings.
How the work is checked
- A real run and client request succeed in the selected environment.
- Model, format, license, runtime, and relevant settings are identifiable.
- The benchmark distinguishes loading overhead from repeated inference.
- Quality and resource conclusions use actual observations with stated limits.
- The final decision does not equate local execution with guaranteed privacy.
Six guided hours include runtime configuration, actual operation, demonstrations, practice, feedback, and questions. Breaks and bulk downloads are outside the total. Ollama is hands-on; llama.cpp is a sourced comparison, not a second required installation. A lab machine can be remote, but the learner operates its runtime rather than a provider's hosted-model API. Training, fine-tuning, production serving, and fleet management are outside scope. Exact hardware needs depend on the verified model and precision. Stage downloads and verify the CPU or GPU path before delivery; recorded traces are not equivalent to local execution.
When another approach is better
- Choose hosted inference when local resource needs or operating effort exceed the task's benefits.
- Do not treat local execution as a guarantee of privacy; inspect logs, network access, retained data, and licenses.
How we train
Contact us for full course details, including duration, delivery options and lab requirements.
Ask about exercises, instructor feedback and the prior knowledge you need. Tool-specific courses marked provisional may become modules in a broader course.
Discuss your learning goalsRelated Courses
These courses list this course as preparation.