Fictional client — portfolio build · targets, not field results
AI Rostering: Accept, Edit, Override — and Log Why
The model proposes 94% occupancy. Exactly one thing is wrong with it.
The Audience
Planning and operations teams adopting AI scheduling — plus anyone designing human-in-the-loop processes.
Responsibilities
Constraint-violation design · reason-audit scoring · model-feedback framing — judgment as an auditable practice.
The Build
Scenario design · audit mechanics · AI-collaboration framing · AI-assisted drafting (disclosed)
How I used AI in this case
AI drafted schedule-proposal variants and reason-log phrasings; constraint-violation design, audit scoring and the human-in-the-loop framing were authored by me. Edited, verified and disclosed.
LO1 · Apply
Accept, edit or override each weekly AI roster proposal — based on its one hidden flaw, not its 94% occupancy.
LO2 · Discriminate
Log a reason specific enough to survive an audit trail: 'the roster looks fine' is a vibe, not a reason.
LO3 · Perform
Keep override frequency low enough that the model — and the team — keep learning.
By the end of this experience a learner can do the three things above — assessed by the judgment-layer debrief, which audits your reasons, not your choices. Every interaction in the build exists to create the evidence for one of them.
Four weeks, four different model failures — and one of them isn't a failure at all:
Full trust at 94% and manual override of everything are the same failure in opposite directions: neither leaves a trail the next planner — or the model — can learn from.
94%
occupancy built on a trainer who cannot be in two cities — the model is blind to geography
6/5
sessions against a contract cap — the model is over-eager, a different disease
The problem was the client's. These were mine — the design and build constraints that shaped the solution.
The AI must be nearly right
A copilot that is obviously wrong teaches nothing. Proposals had to be genuinely good — with one hidden constraint violation that only audit reasoning catches.
Auditing reasons, not clicks
ACCEPT/EDIT/OVERRIDE are noise; the debrief scores the reason log: copy-paste reasoning, silent accepts and reflex overrides all cost.
Judgment as practice
The design treats AI collaboration as an auditable skill — a corrected board with a stated reason outranks a lucky accept.
A planner board with an AI copilot: each week shows the proposal, one hidden constraint violation, and three buttons — ACCEPT / EDIT / OVERRIDE — with a required reason log. The debrief audits the REASONS: copy-paste reasoning, silent accepts and reflex overrides all cost; a corrected board with a specific reason scores highest.
The build itself, not a video about it — open it, make the calls, and read what each one costs.
From human-in-the-loop analysis to a planner board — the stages below are the actual production sequence.
Mapped from four weeks of model failures — one of which isn't a failure at all. Full trust at 94% and manual override of everything are the same failure in opposite directions: neither leaves a trail anyone can learn from. The reason log is the map's spine.
The instructional-design foundations that informed this build:
Action Mapping (Clark)
Every behavior the kit produces is traced back to a business goal and a specific practice activity. See the linked action map for the full chain.
Cognitive Load Theory
The Three.js scene reduces extraneous load by visualising abstract systems. Learners focus on the decision, not the interface.
Situated Cognition
The environment matches the real context. Practice happens where the skill will be performed.
Kirkpatrick L1-L4
Evaluation designed for all four levels: reaction, learning, behavior, results. See the linked evaluation plan.
Pre-committed targets (agreed with stakeholders before build, not retrofitted):
Planners rate the board ≥4/5 for 'our tool, minus the excuses'
L1 · Reaction
Planners rate the board ≥4/5 for 'our tool, minus the excuses'
L2 · Learning
≥75% correct call and auditable reason by week 4
L3 · Behavior
Override/accept decisions in the live tool carry written reasons
L4 · Results
Model constraint reports rise — the loop starts feeding itself
The AI planning rollout had no failure mode for trust: some planners accepted 94% occupancy rosters blindly, others overrode everything and the model never learned. The missing layer was judgment WITH an audit trail. The proposal was a planner board with an AI copilot: four weeks, one hidden constraint flaw each, and a required reason log — with the debrief auditing the reasons, not the choices, because the log is what the next planner and the model owner inherit.
AI-tool feature training considered · rejected
The vendor's default onboarding. Rejected because the planners could already click every button — the failures were judgment failures (silent accepts, reflex overrides), and no feature tour builds the habit of writing down WHY. The buttons were never the gap.
A 'human approves everything' policy considered · rejected
Safe-sounding governance. Rejected because blanket approval without reasons is automation theatre: the model receives no signal, the planner learns nothing, and next month's 94% gets fixed by hand all over again. Edit-fixes-the-week; the flag fixes the model.