AI curriculum
Course outline for Reliable Workflow Execution
Choose what the model and harness coordinate and what an external orchestrator must track. Design branches, joins, loops, and completion checks. Execute work across worker failures, uncertain effects, and restarts. Then build a controlled improvement loop that tests proposed workflow changes before adopting them. Longer agent runs and more iterations do not guarantee completion or improvement.
Reliable Workflow Execution Course Objectives
- Compare model/harness-managed work with externally orchestrated execution.
- Assign planning, scheduling, persistence, verification, and recovery responsibilities explicitly.
- Implement conditional paths, joins, bounded loops, and terminal outcomes.
- Schedule valid dependency stages with capacity and artifact checks.
- Test retries, duplicate effects, lost acknowledgements, and uncertain completion.
- Resume interrupted work and reject stale worker commits.
- Execute compensation and preserve unresolved irreversible outcomes.
- Distinguish agent loops, workflow loops, retries, and improvement loops.
- Run a versioned improvement cycle with independent evaluation and rollback.
- Stop on no progress, exhausted budget, regression, or missing authority.
Pre-requisites
- Basic Python, tests, typed artifacts, and process/concurrency concepts.
- Event-Driven Workflow Design or equivalent durable messages, duplicate handling, and state-transition knowledge.
- Agentic AI Foundations or equivalent actual bounded model/tool operation; a prepared adapter is supplied.
- Workflow Foundations for AI Automation or equivalent workflow, branch, stage, and decision concepts.
- No production orchestrator or distributed-cluster administration experience is required.
Before this course
Complete the required courses or bring equivalent experience. Recommended courses add useful depth; they are not enrolment gates.
Required preparation
Readiness checks
- Basic Python, tests, typed artifacts, and process/concurrency concepts.
- Event-Driven Workflow Design or equivalent durable messages, duplicate handling, and state-transition knowledge.
- Agentic AI Foundations or equivalent actual bounded model/tool operation; a prepared adapter is supplied.
- Workflow Foundations for AI Automation or equivalent workflow, branch, stage, and decision concepts.
- No production orchestrator or distributed-cluster administration experience is required.
Basic Python, tests, typed artifacts, and process/concurrency concepts.
Find your place in the learning mapLab Setup
Delivery requirements: this environment, access, and the exercise materials must be prepared and verified before delivery. This outline does not certify that a lab package is ready.
Stable Python, SQLite, tests, two worker processes, prepared bounded agent harness, authorized model/tool endpoint, disposable artifact directory, queue/storage adapters, local effect/reservation services, and independent inspection commands. Provide persistent branch/iteration state, failure hooks, development/evaluation/ confirmation case separation, baseline config, and a rollback mechanism. Teach the model-versus-harness boundary before the controller comparison. Verify stable dependencies, session recovery, quotas, inference cost, process termination, transaction/fencing behavior, and all lab outputs before delivery. No production publication, real customer actions, cloud cluster, or Temporal server is required.
Detailed Course Outline
Module 1: Choose who coordinates the work (2 hours)
- Model and harness roles: Model-selected steps versus harness execution, state, tools, and limits.
- Native harness capabilities: Native planning, delegation, compaction, and multi-session handoffs.
- Long-horizon limits: Improving long-horizon capability versus independently verified completion.
- External orchestrator: External orchestrators: separate lifecycle, durable state, scheduling, and recovery.
- Existing and custom controllers: Using an existing orchestrator versus designing a small controller.
- Hybrid ownership: Hybrid ownership: bounded agent stages inside an externally managed workflow.
- Actual orchestration comparison: A real harness-managed run and externally coordinated comparison.
- Coordination overhead: Coordination overhead, task evidence, and removing unnecessary scaffolding.
- Guided-time allocation: 0.5 hours for explanation and a worked example, 0.75 hours for guided and independent practice, 0.5 hours for checks and feedback, and 0.25 hours for questions and contingency.
Practical outcome: Run both approaches and assign completion, state, timeout, and recovery ownership without assuming either always succeeds.
Module 2: Design choices, repetition, and termination (2 hours)
- State machines and DAGs: Workflow state machines versus acyclic dependency graphs.
- Conditional routes: If/else routes, explicit conditions, defaults, and unknown evidence.
- Branches and joins: Exclusive alternatives, parallel branches, joins, and skipped-stage semantics.
- Persistent iteration: Fixed iteration, while/until conditions, and persistent iteration identity.
- Different loop purposes: Agent tool loops, retries, and quality-revision loops as different structures.
- Loop termination: Loop exit, no progress, oscillation, cancellation, and budget exhaustion.
- Checked transition proposals: Model-proposed conditions checked before state transitions or effects.
- Outer loop and inner graph: A bounded outer loop around per-iteration acyclic execution.
- Guided-time allocation: 0.5 hours for explanation and a worked example, 0.75 hours for guided and independent practice, 0.5 hours for checks and feedback, and 0.25 hours for questions and contingency.
Practical outcome: Execute both conditional paths and a bounded loop, including nonconvergence and restart mid-iteration.
Module 3: Schedule only work that is ready (2 hours)
- Typed stage contracts: Stage ownership, typed inputs/outputs, and dependency edges.
- Dependency validation: DAG validation, missing inputs, cycles, and ready-stage selection.
- Branch eligibility: Selected branch eligibility and joins that exclude skipped alternatives.
- Parallel capacity: Parallel workers, capacity limits, and separate output paths.
- Accepted artifacts: Artifact publication, checksums, versions, and accepted completion.
- Scheduler extension: A learner-extended scheduler over prepared durable storage.
- Guided-time allocation: 0.5 hours for explanation and a worked example, 0.75 hours for guided and independent practice, 0.5 hours for checks and feedback, and 0.25 hours for questions and contingency.
Practical outcome: Run independent report stages concurrently and assemble only the selected branch's verified artifacts.
Module 4: Separate attempted work from successful effects (2 hours)
- Durable work and acknowledgements: Durable pending work, claims, acknowledgements, and redelivery.
- Attempt and effect identity: Attempt identity versus stable stage, iteration, and effect identities.
- Retry policy: Retryable failures, permanent failures, bounded backoff, and budgets.
- Timeout and uncertainty: Execution timeout, heartbeat timeout, and uncertain external completion.
- Sink idempotency: Sink-side idempotency keys and atomic duplicate recognition.
- Lost-acknowledgement tests: Lost-acknowledgement tests with idempotent and non-idempotent sinks.
- Guided-time allocation: 0.5 hours for explanation and a worked example, 0.75 hours for guided and independent practice, 0.5 hours for checks and feedback, and 0.25 hours for questions and contingency.
Practical outcome: Reconcile an uncertain effect and stop exhausted retries rather than blindly repeating an action.
Module 5: Recover the workflow rather than trust a summary (2 hours)
- Checkpoints and branches: Checkpoints, branch choices, iteration counters, and artifact references.
- Input versions: Input versions, safe write boundaries, and partial outputs.
- Actual process restart: Actual worker termination and restart from persisted state.
- Recovery states: Completed, running, failed, skipped, and uncertain recovery states.
- Independent completion: Independent completion checks versus model-declared success.
- Corrupt artifacts: Artifact corruption, downstream invalidation, and retained accepted outputs.
- Guided-time allocation: 0.5 hours for explanation and a worked example, 0.75 hours for guided and independent practice, 0.5 hours for checks and feedback, and 0.25 hours for questions and contingency.
Practical outcome: Resume interrupted execution with valid branch/loop state and independently verified checkpoint reuse.
Module 6: Keep ownership valid across workers (2 hours)
- Multiple workers: Multiple worker processes sharing queue and workflow state.
- Claims and leases: Claims, ownership versions, leases, and expiry assumptions.
- Stale completion rejection: Compare-and-set completion and stale state-update rejection.
- Effect-side fencing: Effect-side fencing versus local state-version protection.
- Reclaim and restart: Pause, reclaim, restart, and competing completion tests.
- Controller ownership: Controller/harness ownership boundaries and conflicting recovery commands.
- Guided-time allocation: 0.5 hours for explanation and a worked example, 0.75 hours for guided and independent practice, 0.5 hours for checks and feedback, and 0.25 hours for questions and contingency.
Practical outcome: Reclaim work and reject stale completion while explaining the limits of external-effect protection.
Module 7: Correct what execution cannot undo (2 hours)
- Compensation and rollback: Compensation versus rollback, cancellation, retry, and reconciliation.
- Corrective action failures: Business-specific corrective actions and their own failure paths.
- Irreversible effects: Reservation release, partial completion, and irreversible effects.
- Manual resolution: Manual-resolution states and visible incomplete outcomes.
- Failure injection: End-to-end failure injection across worker and service boundaries.
- Architecture recommendation: Native harness, external controller, and simpler-script recommendations.
- Guided-time allocation: 0.5 hours for explanation and a worked example, 0.75 hours for guided and independent practice, 0.5 hours for checks and feedback, and 0.25 hours for questions and contingency.
Practical outcome: Execute a corrective action and explain why another effect needs unresolved/manual status.
Module 8: Engineer a workflow that tests its own improvements (2 hours)
- Loop engineering: Loop engineering as a designed feedback-and-improvement workflow.
- Outcome-driven proposals: Outcome observations, failure analysis, and bounded change proposals.
- Artifact and workflow improvement: Artifact refinement versus changing the workflow for future runs.
- Versioned changes: Versioned prompts, configurations, policies, and tool changes.
- Independent evaluation: Development cases, frozen evaluation, independent checks, and regression guards.
- Adoption and rollback: Candidate/baseline comparison, adoption, rejection, and rollback.
- Improvement stop rules: Cost, quality, no-progress, oscillation, and explicit stopping criteria.
- Bounded improvement prototype: A bounded executed prototype of continuous improvement across runs.
- Required improvement core: One required candidate evaluation and rollback; further cycles are optional.
- Guided-time allocation: 0.5 hours for explanation and a worked example, 0.75 hours for guided and independent practice, 0.5 hours for checks and feedback, and 0.25 hours for questions and contingency.
Practical outcome: Run a candidate-change evaluation cycle and adopt or reject it from checked evidence while preserving a rollback version.
Orchestration comparison
- Next work — Model choices within harness capabilities and limits — Controller selects eligible stages; a stage can contain an agent
- State and progress — The harness's actual session/checkpoint facilities — Explicit workflow/stage state outside a particular agent session
- Long waits and restart — Only what the selected harness demonstrably supports — Implemented durable timers/waits and recovery policies, if provided
- Completion — Independently check goals/artifacts, not only a final message — Independently check accepted stage outputs and terminal workflow criteria
- Failure handling — Harness-specific retry, handoff, and context behavior — Explicit retry, ownership, uncertainty, and corrective-action policies
- Improvement — Can host revision/evaluation loops — Can coordinate and persist the same loops across runs
- These are responsibility questions, not universal product guarantees. A harness can already contain durable orchestration; an external controller can itself use agents. External means a separately managed execution boundary, not necessarily a remote service or distributed cluster. A poor controller can be less reliable than an established harness. Reassess extra coordination as models/harnesses improve.
Four different loops
- **Agent loop:** observe, select a tool/action, execute, and inspect feedback within a task. The harness implements the loop around model choices. - **Workflow control loop:** repeat a defined set of stages until an explicit condition, count, deadline, or budget ends it. It can contain no model at all. - **Retry loop:** reattempt failed work under a failure policy. Retrying unchanged work is not, by itself, evidence of learning or improvement. - **Improvement loop:** observe outcomes, propose a change, test it against a baseline, and adopt or reject it. The next run uses an accepted version.
- Here, loop engineering means designing the feedback, state, evaluation, change, and stopping rules that make repeated work useful. It is not a synonym for the agent tool loop. A draft/evaluate/revise cycle can improve one artifact; a wider loop can improve the workflow's prompt, tool, configuration, or review policy for future tasks. Neither automatically updates model weights or guarantees progress. An improvement loop can use either orchestration approach or combine them.
Practical exercise
- Use a synthetic operational-report task with typed source, calculation, evidence, draft, check, and registration artifacts. First run it through a prepared harness whose real model selects read/calculation tools. Inspect actual artifacts and a completion checklist independently. Introduce a bounded early-stop or omitted-step condition through a test hook; do not depend on a live model failing on demand. The failure fixture supplements, rather than replaces, the actual harness run.
- Run the same task with a separate Python/SQLite controller using two actual worker processes. The starter supplies storage, queue, model adapter, controlled services, and inspection commands. Learners extend scheduling, branch/loop policy, retry decisions, and accepted completion. Compare actual state ownership, completion, cost, and recovery evidence; do not claim a short lab establishes which architecture handles every long-horizon task. Temporal is a sourced existing-orchestrator comparison, not an unannounced required second runtime.
- Implement an if/else path: complete evidence permits drafting; missing mandatory evidence routes to a documented wait/escalation outcome. Include parallel valid input stages and a join that does not wait for an intentionally skipped alternative. Implement draft/check/revise with a fixed maximum iteration count, persisted iteration identities, and explicit acceptance/no-progress rules. Test success, missing/unknown conditions, exhaustion, repeated identical candidates, oscillating outcomes, cancellation, and restart mid-loop. A known DAG for one iteration stays acyclic; the outer state machine owns repetition. Do not insert a back edge into a DAG and call its validation failure a prohibition on all workflow loops.
- Kill and restart a worker. Test transient errors, exhausted retries, partial output, corrupt artifacts, and acknowledgement loss after sink registration. Check service state independently. Claim one committed effect only where the sink atomically enforces its stable effect key. With the non-idempotent variant, preserve unknown status and reconcile rather than declaring blind retry safe. Persist model outputs and accepted choices where needed; replay must not silently resample completed work.
- Pause a worker, reclaim its stage, and reject its later state update. Explain or test the supplied effect-side fencing separately. Run reserve-then-fail and release compensation, including a failed compensation that remains unresolved. Cancellation or deleting a local row cannot erase an effect someone already saw.
- Finally execute a workflow-improvement prototype. Freeze a baseline prompt/config, quality criteria, budget, and regression checks for report drafting. Collect failures on development cases. A real model proposes a bounded prompt/config change using that feedback. The improvement driver runs the candidate and baseline on the same evaluation inputs, checks factual/numeric correctness independently, and compares quality, cost, and review effort. Version inputs, candidate, outputs, and the adoption decision. The candidate cannot edit the evaluator, reference answers, permissions, or its own acceptance threshold.
- The required improvement core is one actual model-proposed candidate, one candidate/baseline evaluation, an adoption or rejection decision, and one rollback using a supplied regressing candidate. Further cycles are optional instructor demonstrations or extensions, not additional compulsory work. Keep final confirmation cases out of proposal feedback. Rejection is a valid result. A model checker can share the generator's blind spots. This finite prototype does not establish indefinite self-improvement.
- Reserve one branch, loop-exit, or failure test for independent learner design before explanatory feedback. Report actual runs and explicit unresolved outcomes, not merely a recovery narrative or the agent's claim that everything is complete.
- Learners extend the scheduler, branch/loop policy, retry decisions, and accepted-completion checks. They run the supplied parameterized failure suite and author one independent branch, loop-exit, or failure test. Storage, queues, model adapters, sinks, and effect-side fencing examples are prepared infrastructure.
Delivery and timing
- Guided hours include explanations, demonstrations, learner practice, checks, feedback, questions, and contingency as allocated above. Breaks, bulk downloads, and prerequisite account/environment setup are additional. These allocations require a timed pilot before delivery; they are not measured completion times.
What you will produce
Execute branches and loops reliably, recover interrupted work, and evaluate a bounded workflow improvement.
How the work is checked
- Both orchestration approaches execute with independently checked completion.
- Planning, scheduling, persistence, action, and recovery owners are explicit.
- Conditional paths and joins handle selected, skipped, and unresolved branches.
- Loops persist iteration state and stop on the configured terminal conditions.
- Inner-agent, retry, workflow-control, and improvement loops are distinguished.
- Actual interrupted execution resumes with checked artifact/branch/loop state.
- Retry and uncertain-effect decisions reflect actual sink guarantees.
- A stale worker cannot commit completion under the tested ownership policy.
- Compensation runs and unresolved irreversible/corrective failures stay visible.
- An actual improvement proposal is evaluated against the preserved baseline.
- Adoption/rejection and rollback obey frozen checks, budgets, and permissions.
- Results disclose small-sample, evaluator, timing, and model-variation limits.
16 guided hours. A prepared harness and separate local Python/SQLite controller execute the same bounded task. Learners extend scheduling, branch/loop policy, retry decisions, and accepted completion; storage, sinks, model adapters, failure hooks, and effect-side fencing demonstrations are supplied. The improvement core is one candidate evaluation and one rollback exercise; further cycles are optional. No exactly-once, guaranteed completion, unbounded self-improvement, or automatic model-weight training claim is made.
When another approach is better
- A simple sequential script may suffice when work is short and effects are reversible.
- Do not use a DAG to hide business cycles or assume that compensation reverses every external effect.
How we train
Contact us for full course details, including duration, delivery options and lab requirements.
Ask about exercises, instructor feedback and the prior knowledge you need. Tool-specific courses marked provisional may become modules in a broader course.
Discuss your learning goalsRelated Courses
These courses list this course as preparation.