# AI Policy in Practice — Exercise workbook AIPEX | Edition 2026-10-13 · reviewed edition A decision briefing and delivery plan for a fictional AI recruitment policy case. Use fictional or anonymised examples. Course checks and saved progress are available at https://theaipex.org/courses/ai-policy-in-practice ## A policy problem, not a technology ### Teaching and case materials #### Start with a decision An AI policy team earns its keep by helping someone make a better decision. “What should we do about AI?” is an area of interest. It is not yet a workable assignment. Name the decision-maker, the people affected, the harm or opportunity, and the decision that can actually be taken. Throughout this course you advise the **fictional Northbridge Employment Partnership**. It funds local employment services and has heard complaints about automated recruitment screening. Its chair wants a recommendation in six weeks. You have two analysts and a small research budget. All organisations, numbers and incidents in this case are invented for learning; they are not findings about a real council or supplier. Your initial case file contains three claims: employers report too many applications; applicants say rejections are difficult to challenge; suppliers say screening improves consistency. These claims can all be true. Your task is not to choose which group you like most, or to decide whether AI is good in general. #### Describe the mechanism Follow a concrete pathway: a candidate submits an application; a screening system scores it; someone uses that score to decide who gets interviewed; a rejection affects the candidate's opportunities. Ask where a preventable problem could enter that pathway. A model can be accurate at predicting historic hiring decisions while reproducing an employer's past mistakes. A transparent rejection can still be unfair. A slow manual process can also exclude people. Distinguish **the observed problem**, **your explanation**, and **the response**. “Applicants cannot obtain a review” describes an experience. “Employers rely on automated scores without a challenge route” is a hypothesis about why it happens. “Require a review mechanism” is an intervention to test. Mixing these together makes your preferred policy appear inevitable. #### Set boundaries Choose a population, use case, place and time horizon. For Northbridge, begin with screening for jobs advertised through partnership-funded services over the next twelve months. This is narrower than regulating every recruitment product in the UK, but it gives the partnership a plausible lever. Record the wider questions you are setting aside, rather than pretending they do not matter. A useful decision question is: **What conditions, if any, should Northbridge attach to funded recruitment services to improve access to review without making the service unusable?** That question leaves several answers open. Carry it into your research rather than starting with a demand to ban or subsidise a technology. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Three documents, three different claims The chair’s email asks for “fairer screening within funded services”, not a new national regulation. Three documents accompany it: | Document | What it contains | What it leaves out | | --- | --- | --- | | Adviser note | Six accounts of being unable to obtain a rejection reason | How applicants were selected; prevalence | | Provider log | 18 review requests among 600 applications | Phone requests; rejection count; unmet need | | Supplier brochure | Faster-shortlisting claim | Comparator, complete workflow cost and challenge outcomes | None of these establishes regional prevalence or which actor caused a failure. The accounts justify investigating an access problem; the log tells you about recorded demand in one process; the brochure offers a claim to test. Keep that distinction when writing your scope. Ask whether the alleged barrier also exists in manual recruitment. If it does, a technology-only policy could miss the mechanism. #### Connect the reading to your practice Use the Green Book's objectives-led appraisal alongside the Service Standard's user-needs perspective. Before proposing a recruitment tool, ask what would improve applicants' experience and what Northbridge can actually decide. Sources for this comparison: [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026); [Government Digital Service: Service Standard](https://www.gov.uk/service-manual/service-standard). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Write a 150–200 word scoping note for Northbridge. Include the decision-maker, population, observed problem, a causal hypothesis, the decision deadline and one explicit boundary. End with a decision question that leaves at least two responses open. Use the three documents to name one supported observation, one untested explanation and one missing denominator. Keep your note to 150–200 words. ### Review criteria - Names a decision Northbridge can take - Separates an observation from an explanation - Defines who and what is in scope - Does not build a preferred intervention into the question - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026) — Read the rationale and objectives discussion. Rewrite Northbridge's assignment without assuming an AI solution. - [Government Digital Service: Service Standard](https://www.gov.uk/service-manual/service-standard) — Read points 1 and 2 on user needs and whole problems. Identify an applicant need missing from the supplier's framing. ### Your answer [Write your response here] ## Objectives and a theory of change ### Teaching and case materials #### Decide what better means An objective should describe a change in the world, not just an activity. “Publish guidance” is an output. “Applicants can obtain a timely, meaningful review” is an outcome. Publication may contribute to that outcome, but only if the right people read the guidance, have incentives to follow it and can act on it. For Northbridge, define improvement across three dimensions: effective access to review, equitable treatment of applicants and a recruitment service employers can still use. Resist collapsing these into one headline score. A quicker review process that never changes a decision may be less useful than a slower one that identifies serious errors. #### Build a causal chain Try a provisional intervention: funded providers must explain how screening is used and offer a review by a trained person. The causal chain is: a condition in the funding agreement changes provider procedures; providers give understandable notices; applicants know a review exists; applicants can request it without unreasonable effort; reviewers have information and authority; correctable errors are identified and remedied. Write the weakest link under each arrow. Applicants may fear that a complaint harms future job prospects. Reviewers may rubber-stamp scores. Providers may lack records. Employers may leave the scheme. These are design questions and evaluation questions, not reasons to ignore the chain. #### Specify measures before targets Possible measures include the proportion of eligible applicants who know about review, time to a reasoned response, reviews identifying a material error, applicant experience, and employer participation. State denominators. “Ten complaints” means little without knowing whether there were fifty applicants or fifty thousand. A fall in complaints could mean fewer problems or a harder complaint process. Do not invent a baseline as if it were known. Mark it “to establish” and say how: a sample of recent applicants, provider process checks and interviews with people who did not complain. In this fictional exercise, you can choose illustrative targets, but label them as proposals needing evidence and stakeholder agreement. #### Make trade-offs visible An objective can conflict with another. A very fast review may require more staff; an extensive notice may overwhelm applicants. Identify minimum conditions you will not trade away casually, alongside benefits that can be balanced. Give each objective an owner and a review date. Your theory of change should help a delivery team decide what to test, and help a chair understand where confidence ends. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Find the broken link In a small process test, all 40 sampled rejection notices mention review. Twelve applicants can explain how to request it. Six request a review; all six receive replies on time, but reviewers cannot access the screening inputs and merely restate the score. These are test observations, not a representative baseline. Notice delivery is 40/40; demonstrated understanding is 12/40. Timeliness among received requests is 6/6. None measures whether a decision was meaningfully reconsidered. A target of “100% timely replies” could be met while the central mechanism fails. Map understanding and reviewer access separately; a single completion indicator conceals both. #### Connect the reading to your practice The Magenta Book supplies the theory-of-change method; the OECD principles supply a different lens on desirable AI outcomes. A causal chain and a statement of values do different jobs. Test whether your proposed mechanism supports the outcome you chose. Sources for this comparison: [HM Treasury: The Magenta Book (2026)](https://www.gov.uk/government/publications/the-magenta-book/magenta-book-central-government-guidance-on-evaluation-html); [OECD: AI Principles](https://www.oecd.org/en/topics/ai-principles.html). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Draft three outcome objectives and a six-link theory of change for the review requirement. For each objective propose a measure and identify a missing baseline. Include two assumptions that could break the chain. In your theory of change, place these three measures at different links. Identify the failed link and add an evidence test for meaningful review; do not label the six replies as six successful outcomes. ### Review criteria - Objectives concern outcomes rather than publication or meetings - Measures have clear denominators or definitions - The causal chain explains behaviour - At least two plausible failure mechanisms are identified - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [HM Treasury: The Magenta Book (2026)](https://www.gov.uk/government/publications/the-magenta-book/magenta-book-central-government-guidance-on-evaluation-html) — Read section 2.2.1, Understand the Theory of Change. Mark a weak causal link and an assumption in your Northbridge theory of change. - [OECD: AI Principles](https://www.oecd.org/en/topics/ai-principles.html) — Read the values-based principles. Choose a relevant outcome and explain how you would observe it; these principles are not a substitute for domestic law. ### Your answer [Write your response here] ## Audit an evidence claim ### Teaching and case materials #### What exactly was measured? The fictional supplier SelectWise tells Northbridge that its product is “92% accurate” and “saves 40% of staff time”. Before repeating either number, ask for the task, sample, metric, comparator, period and uncertainty. Accurate at what: reproducing a historic shortlist, identifying later job performance, or finding all candidates qualified for an interview? Imagine a test set with 1,000 applications, of which only 80 meet an agreed interview threshold. A system that rejects everyone is 92% accurate if accuracy simply counts correct classifications. It is useless for finding candidates. This arithmetic is illustrative, not a claim about real recruitment systems. Ask how many qualified applicants are missed, how many unqualified applicants are advanced, and how the threshold changes those errors. #### Separate four questions First, does the technical result hold in the test setting? Second, does it transfer to Northbridge's applicants and jobs? Third, does it improve the actual recruitment workflow against a credible alternative? Fourth, does that improvement justify the costs and distribution of harms? A strong answer to the first is not a shortcut to the fourth. Check provenance: who commissioned the study, who chose the comparator and whether the methods and limitations are available. Supplier evidence is not automatically worthless; it needs independent scrutiny proportionate to the decision. Applicant testimony is not a population estimate, but it can identify mechanisms that a benchmark misses. #### Examine the counterfactual “Saves time” must be compared with something. Was the comparator an old manual system, an improved manual process, or a different tool? Does the time estimate include procurement, training, checking, appeals and fixing failures? A saving in one team can create work elsewhere or shift effort onto applicants. Sample selection matters. A trial with enthusiastic staff, easy roles and English-language applications may say little about difficult cases. A before-and-after comparison may confuse the product with a change in vacancy mix. Ask what would have happened without the intervention, and what alternative explanations remain. #### Record confidence without theatre Build a claim log: claim, evidence, limitations, relevance, confidence and decision implication. Use plain statements such as “promising in the supplier's test population; transfer to local vacancies untested”. Precision in wording is more useful than a decorative traffic light with no reasoning behind it. The next action should be clear: request subgroup error rates, commission a workflow trial, or discount an unsupported claim. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Read the supplier's confusion matrix An independent label means “meets the agreed interview threshold”, not future job success. On the same fictional 1,000 applications: | | Meets threshold | Below threshold | | --- | ---: | ---: | | System advances | 60 | 20 | | System rejects | 20 | 900 | Accuracy is (60 + 900)/1,000 = 96%. Recall is 60/80 = 75%: a quarter of candidates meeting the threshold are missed. Precision is 60/80 = 75%: a quarter of those advanced are below threshold. These answer different operational questions. The always-reject baseline has 92% accuracy and zero recall. Compare against trained manual screening too, and examine whether the labels reflect relevant, defensible criteria. A real-world comparison: the [ICO’s 2024 recruitment audits](https://ico.org.uk/media2/migrated/4031620/ai-in-recruitment-outcomes-report.pdf) identified gaps in accuracy testing and problems using inferred demographic information for bias monitoring. These selected audits excluded generative AI and do not estimate market-wide prevalence. Apply that scrutiny to the fictional labels and group evidence below. Group A has 16 missed candidates among 40 meeting the threshold; group B has four among 40. False-negative rates are 40% and 10%. This descriptive gap needs investigation of uncertainty, label quality and mechanisms; it is not by itself a legal finding. #### Connect the reading to your practice Model Cards proposes structured reporting of intended uses and evaluation. NIST's MEASURE function asks how risk is assessed in context. Use both to interrogate a supplier's accuracy claim rather than treating a headline score as a deployment decision. Sources for this comparison: [Mitchell et al.: Model Cards for Model Reporting (2019)](https://arxiv.org/abs/1810.03993); [NIST AI Risk Management Framework: Core](https://airc.nist.gov/airmf-resources/airmf/5-sec-core/); [ICO: AI in recruitment audit outcomes (2024)](https://ico.org.uk/media2/migrated/4031620/ai-in-recruitment-outcomes-report.pdf). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Produce an evidence audit of the supplier's two claims. Ask six specific questions, calculate the reject-everyone accuracy in the case example, and explain what evidence would change your decision. Add accuracy, recall, precision and both group false-negative rates to your audit. Explain why a 96% headline does not resolve the policy choice. Keep the six evidence questions and the reject-everyone calculation. ### Review criteria - Identifies metric and denominator - Checks a credible comparator and full workflow costs - Considers transfer and selection - Links missing evidence to a decision - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [Mitchell et al.: Model Cards for Model Reporting (2019)](https://arxiv.org/abs/1810.03993) — Read the model-card reporting framework, especially intended use and evaluation. List three disclosures missing from Northbridge's supplier claim. - [NIST AI Risk Management Framework: Core](https://airc.nist.gov/airmf-resources/airmf/5-sec-core/) — Read MEASURE 2.1, 2.3 and 2.5. Ask whether the test population and conditions support the proposed recruitment use. - [ICO: AI in recruitment audit outcomes (2024)](https://ico.org.uk/media2/migrated/4031620/ai-in-recruitment-outcomes-report.pdf) — Read the executive summary and fairness findings of the 2024 consensual audit. Identify a concrete evidence question for recruitment providers. These selected audits do not estimate market-wide prevalence or establish the current ADM rules. ### Your answer [Write your response here] ## Research under uncertainty ### Teaching and case materials #### Not every uncertainty deserves another study You have six weeks, not a mandate to resolve every AI debate. Start with your decision and ask which unknowns could alter it. A missing fact about whether Northbridge's funding agreement can include a review condition may matter more than another forecast of AI's long-run economic effects. Rank uncertainties by decision sensitivity and whether they can be reduced in time. For the recruitment case, identify legal authority, prevalence and severity of review failures, provider capacity, applicant trust and the likely participation response. Record what you already know, what you infer, and what remains genuinely unknown. #### Triangulate evidence Design a modest research package: review the funding agreement with appropriate legal expertise; interview providers; sample recent applicant experiences; inspect a few screening workflows; and ask suppliers for relevant testing documentation. Different sources answer different questions. Interviews explain mechanisms but may miss people who are hardest to reach. Administrative data shows recorded events but excludes experiences never recorded. Include a dissent search. If your emerging recommendation is a review requirement, look for cases where such requirements became inaccessible paperwork. Ask what a critic would need to demonstrate to make you change course. Document your search boundaries and non-response so a later reader can assess the evidence rather than just trust your confidence. #### Use scenarios as stress tests Choose two or three plausible operating conditions: providers have enough capacity; review demand exceeds capacity; employers withdraw from the funded network. These are not probabilities unless you have a defensible basis for assigning them. Ask whether your option remains useful in each, what early signal would distinguish them, and what adaptation would be available. A robust recommendation may be a limited pilot with explicit conditions, even if you cannot rank every future outcome. Reversibility is valuable when learning is possible and exposure can be bounded. It is not a substitute for safeguards where harm would be severe or difficult to undo. #### Give decision-makers usable uncertainty Avoid both “more research is needed” and false certainty. Say: “We are confident the current review route is hard to navigate in the sampled services; we do not yet know prevalence across all providers. A short discovery phase should establish that baseline before imposing a network-wide condition.” End with a decision that can be taken now and a later decision that better evidence will unlock. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Spend a finite research budget You have ten analyst-days this fortnight. Reviewing renewal authority takes two days. Observing six review workflows takes four. Recruiting people who abandoned applications takes three. A general employer opinion survey takes eight. A consultant offers a technology forecast after the chair's deadline. Choose a package that could distinguish your leading options. Authority is a feasibility gate; workflow observation tests whether records and reviewers can support the mechanism; absent applicants challenge your sampling frame. Nine days leaves one for synthesis. That is a defensible allocation, not a formula requiring every team to choose the same methods. If authority is already settled, the ranking changes. #### Connect the reading to your practice Compare evaluation planning in the Magenta Book with the range of assurance techniques in DSIT's introduction. A study should answer a decision-relevant question; an audit or impact assessment may answer a different one. Explain what uncertainty your chosen approach would leave unresolved. Sources for this comparison: [HM Treasury: The Magenta Book (2026)](https://www.gov.uk/government/publications/the-magenta-book/magenta-book-central-government-guidance-on-evaluation-html); [DSIT: Introduction to AI assurance](https://www.gov.uk/government/publications/introduction-to-ai-assurance/introduction-to-ai-assurance). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Create a five-row uncertainty register: unknown, why it matters, research method, deadline and decision changed. Add two operating scenarios and one trigger for adapting your recommendation. Give your five-row register analyst-days as well as deadlines. Select work totalling at most ten days, state what you will defer and explain how a contrary finding would change the recommendation. ### Review criteria - Prioritises decision-sensitive uncertainty - Uses complementary research methods - Includes a realistic deadline - Scenario triggers connect to an available action - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [HM Treasury: The Magenta Book (2026)](https://www.gov.uk/government/publications/the-magenta-book/magenta-book-central-government-guidance-on-evaluation-html) — Read the discussion of evaluation purpose and questions in section 2.2.2. Rank Northbridge's uncertainties by whether resolving them could change the recommendation. - [DSIT: Introduction to AI assurance](https://www.gov.uk/government/publications/introduction-to-ai-assurance/introduction-to-ai-assurance) — Read AI assurance mechanisms in section 4.2. Choose an evidence-gathering technique and explain why its result would be useful within six weeks. ### Your answer [Write your response here] ## Institutions and authority ### Teaching and case materials #### A policy without an owner is a wish AI problems cut across institutions. That does not mean every institution can act on every part. Map four roles: who decides, who delivers, who oversees, and who can challenge. Then distinguish formal powers from budget control, contractual leverage, expertise and public influence. Northbridge controls some funding agreements and convenes employers. It does not acquire the powers of a national regulator because it cares about AI. A provider may control staff practice but not the underlying recruitment system. A supplier may control model updates while an employer retains the hiring decision. Responsibilities must be assigned at the level where action is possible. #### Draw the chain of authority Begin with the partnership's governing documents and the actual agreements. Who approves new conditions? Is there a renewal date? Which services and employers are covered? Are there existing obligations relevant to the problem? Ask appropriately qualified colleagues to check the legal position, including applicable employment, equality and data protection requirements. This course teaches how to structure those questions; it does not decide legal compliance for a real organisation. For each proposed intervention, make a row: instrument, authority, decision-maker, affected actor, enforcement route and unresolved questions. A voluntary statement of principles has different effects from a funded-service condition. A contract condition is not a new statute. An awareness campaign cannot compel a supplier to provide logs. #### Coordinate rather than duplicate Identify organisations already addressing the problem and ask what you can add. Relevant public bodies depend on the jurisdiction, sector and specific conduct. Use current official sources to check responsibilities rather than copying an old organisational chart. A referral protocol may be more useful than inventing another reporting portal. International frameworks can organise questions, but they are not interchangeable with domestic powers. A voluntary risk framework can guide evidence requests; it does not create an enforcement mandate. Similarly, a rule that applies to one category of organisation should not be advertised as applying universally. #### Turn the map into action The point is not a handsome diagram. It is knowing whose agreement is needed, where delivery can fail and what Northbridge should refer elsewhere. End the institutional map with an accountable sponsor, an implementation owner and a list of approvals. If authority is uncertain, make resolving it a condition before proceeding, not a footnote after launch. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Read the instrument before promising enforcement Assume Northbridge's existing agreement permits process inspections, requires twelve weeks' notice for new conditions at renewal and contains no power to compel a supplier directly. The board approves renewals; providers contract with suppliers. These are fictional instrument terms, not UK legal rules. A six-week recommendation can request inspection and prepare a renewal condition. It cannot promise a condition takes effect immediately. Supplier information access needs a provider-mediated contractual route, a voluntary arrangement or another established power. DRCF coordination does not supply Northbridge with missing authority. In another jurisdiction, replace the institutions and instruments; retain the method of checking actor, power, scope and challenge route. #### Connect the reading to your practice The DRCF is a voluntary cooperation forum, not a regulator with its own statutory remit. Compare that institutional description with OECD accountability principles. Distinguish coordination, responsibility and legal authority when drawing Northbridge's decision map. Sources for this comparison: [Digital Regulation Cooperation Forum: roles and remit](https://www.drcf.org.uk/about-us); [OECD: AI Principles](https://www.oecd.org/en/topics/ai-principles.html). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Map Northbridge, providers, employers, suppliers and applicants. For a funding condition and a voluntary guidance option, identify the decision owner, instrument, delivery actor, oversight route and one authority question requiring expert advice. Include an authority row for inspection now, a renewal condition later and supplier logs. Distinguish who approves from who must deliver, and identify a claim the chair cannot currently make. ### Review criteria - Distinguishes formal authority from influence - Assigns delivery and oversight - Avoids assuming a local body can regulate nationally - Makes unresolved authority an explicit dependency - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [Digital Regulation Cooperation Forum: roles and remit](https://www.drcf.org.uk/about-us) — Read Who we are, Members and How we work. Separate each member regulator's role from the forum's coordination function; follow remit links before making a real legal claim. - [OECD: AI Principles](https://www.oecd.org/en/topics/ai-principles.html) — Read Accountability. Assign a responsible actor to each Northbridge decision, then identify the separate evidence needed to establish that actor's authority. ### Optional video [NIST: Introduction to the AI Risk Management Framework (2023)](https://www.nist.gov/video/introduction-nist-ai-risk-management-framework-ai-rmf-10-explainer-video) · 6 min 31 sec Use 00:53–01:25 for the framework’s voluntary scope, then 03:33–04:05 for GOVERN, MAP, MEASURE and MANAGE. The recording introduces the 2023 framework; its future-work statements are historical, not a current law or organisation update. Give one Northbridge action an owner under each function. Then name the separate instrument or agreement needed for that actor to act. Which proposed action remains outside Northbridge’s authority? English captions and Show Transcript in the publisher’s player. Reading alternative below. [Reading alternative](https://airc.nist.gov/airmf-resources/airmf/5-sec-core/) ### Your answer [Write your response here] ## Stakeholders and legitimacy ### Teaching and case materials #### Influence is not the same as exposure A stakeholder map should show who can block implementation and who bears its consequences. These groups may be very different. A supplier with a large account team can attend every meeting; a rejected applicant may have neither time nor confidence to join a formal consultation. For Northbridge, include applicants, people who abandoned applications, employment advisers, small and large employers, providers, suppliers, relevant oversight bodies and groups supporting people facing access barriers. Do not treat any group as having a single opinion. People may want both efficient recruitment and a meaningful challenge route. #### Ask questions that matter Engagement is most useful when it can alter a decision. Ask applicants what made review difficult, providers what information they can access, employers which burdens would change their participation, and advisers where existing processes fail. Present real alternatives with trade-offs. “Do you support fair AI?” tells you almost nothing about design. Use accessible formats and proportionate support. A short phone interview, plain-language scenario and translated materials may reveal more than a long online questionnaire. Decide how to handle sensitive experiences and avoid collecting identifiable case information you do not need. Give people a way to contribute without being publicly named. #### Prevent a false consensus Record disagreements explicitly. Applicants may favour independent review; providers may prefer internal review for speed; employers may oppose sharing detailed records. Do not write “stakeholders support the proposal” because the best-resourced attendees agreed. Track who participated, who was missing and how those limits affect confidence. Conflicts of interest should be visible. A supplier can contribute technical expertise while also benefiting from a particular policy. An advocacy group can contribute lived-experience evidence while favouring a particular remedy. Assess evidence and incentives rather than dismissing participation because someone has an interest. #### Close the loop Explain what changed after engagement, what did not and why. A feedback note should connect input to decisions: “Applicants could not understand the original notice, so we tested a shorter version; we retained an internal first review because independent review capacity is limited, with escalation for unresolved cases.” This makes engagement accountable. The goal is a defensible process and a better design, not universal agreement or a photograph of a busy workshop. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Who is absent from the consultation? A workshop attracts eight supplier representatives, six employers and one active applicant. A survey goes only to people who completed an online application. An adviser reports that some people abandoned the process after struggling with language or accessibility. The budget can support eight short interviews and one accessible design session. Attendance is not a vote weighted by institutional headcount. Recruit affected applicants through advisers, including people outside the active list; offer private phone or assisted participation. Report recruitment limits and disagreement. Compensation for time, where feasible and permitted, supports access but does not buy endorsement. Do not infer prevalence from these interviews. #### Connect the reading to your practice Cabinet Office consultation guidance addresses who can participate and when input is useful. Ada, AI Now and OGP's comparative research offers an independent account of public-sector accountability mechanisms. Ask how affected applicants could change the policy, rather than only comment on a finished proposal. Sources for this comparison: [Cabinet Office: Consultation Principles](https://www.gov.uk/government/publications/consultation-principles-guidance/consultation-principles-2018); [Ada Lovelace Institute, AI Now and OGP: Algorithmic accountability for the public sector (2021)](https://www.adalovelaceinstitute.org/report/algorithmic-accountability-public-sector/). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Design a one-page engagement plan with four groups, a decision-relevant question for each, accessible participation, an approach to conflicting interests and a way to report back. Identify a group likely to be missing. Allocate the eight interviews and design session across your engagement plan. State whose view remains missing and write two sentences reporting disagreement without claiming consensus. ### Review criteria - Includes people affected with limited institutional power - Questions could change design - Explains participation barriers - Records disagreement and closes the feedback loop - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [Cabinet Office: Consultation Principles](https://www.gov.uk/government/publications/consultation-principles-guidance/consultation-principles-2018) — Read principles 2, 4, 6 and 7. Design an accessible way to hear from applicants before Northbridge has settled on an option. - [Ada Lovelace Institute, AI Now and OGP: Algorithmic accountability for the public sector (2021)](https://www.adalovelaceinstitute.org/report/algorithmic-accountability-public-sector/) — Read the report's policy examples and lessons. Choose an accountability mechanism and explain what would need adapting to Northbridge; this is historical comparative research, not a map of current law. ### Your answer [Write your response here] ## Build a real options set ### Teaching and case materials #### Options should differ in mechanism Three versions of the same preferred policy do not constitute a meaningful choice. Start with how each option could change behaviour. Information may change understanding; funding conditions may change incentives; a redesigned service may change what is possible; oversight may change consequences for poor practice. For Northbridge, create a longlist: keep existing arrangements with better monitoring; improve the manual review process; publish voluntary screening guidance; attach notice and review conditions to funding; run a restricted service pilot with independent review; or stop funding automated screening in the covered service. Do not assume every option is feasible or desirable. The longlist is a way to reveal alternatives before narrowing them. #### Use a credible baseline The baseline is what would reasonably happen without your new proposal, including changes already committed. It is not a deliberately neglected service that makes the preferred option look brilliant. If provider training is already funded, include it. If contracts renew next year, that timing affects both the baseline and the intervention. Consider a non-AI alternative. If the problem is opaque rejection, a simpler recruitment process, clearer criteria or better staff capacity might help regardless of the technology. Equally, a ban on one system may cause substitution to another process with the same problem. Follow the causal mechanism rather than the label on the tool. #### Filter transparently Shortlist against explicit constraints: authority, relevance to objectives, delivery capacity, affordability and potential harm. Keep a record of why an option was removed. An option should not disappear merely because it inconveniences a powerful stakeholder or contradicts the team's first idea. Separate constraints from preferences. Lack of legal authority may rule an option out for Northbridge. A preference for low administrative burden is a criterion that can be weighed alongside others. If an option needs action from another body, it might remain a referral recommendation rather than a direct intervention. #### Specify the option enough to compare “Human oversight” is not a costable design. Say who reviews, when, with what information, how many cases, what authority and what happens after review. A shortlist of three well-described options is more useful than twenty slogans. Keep the same population and time horizon across them so differences in scope do not silently determine the result. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Prevent a rigged shortlist Provider training is already funded and scheduled; it belongs in every option's baseline. Staff can improve notices and manual review now. A new funding condition needs renewal authority. Independent external review capacity is not yet secured. A prohibition could eliminate one automation pathway, while employers might substitute another opaque process. Write each option as actor → action → behavioural change → applicant outcome. A banner saying “ethical AI” is not comparable to a staffed route. A constraint can be investigated rather than assumed: independent capacity might be acquired later, but it cannot be booked as available now. Keep reasons for exclusion and a condition under which you would reopen an option. #### Connect the reading to your practice Use Green Book options appraisal to compare mechanisms, then use the independent accountability research to widen the set of governance responses. A transparency record, a review route and a procurement condition are different interventions. Do not assume any of them solves the problem on its own. Sources for this comparison: [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026); [Ada Lovelace Institute, AI Now and OGP: Algorithmic accountability for the public sector (2021)](https://www.adalovelaceinstitute.org/report/algorithmic-accountability-public-sector/). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Create a six-option longlist, then shortlist three including a credible baseline. Describe each shortlisted mechanism and explain two exclusions. Include at least one alternative that improves the service without adopting new AI. For each shortlisted option, add a mechanism sentence and a resource or authority dependency. Keep the funded training in all three comparators and say what evidence would reopen one excluded option. ### Review criteria - Options have different behavioural mechanisms - Baseline includes committed changes - Non-AI alternative is credible - Exclusions are reasoned and visible - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026) — Read the longlist and shortlist appraisal material. Construct alternatives that differ in how they address the barrier to review, including a credible non-AI option. - [Ada Lovelace Institute, AI Now and OGP: Algorithmic accountability for the public sector (2021)](https://www.adalovelaceinstitute.org/report/algorithmic-accountability-public-sector/) — Compare two accountability-policy examples in the report. Identify their intended mechanism and one condition that could prevent it working in Northbridge. ### Your answer [Write your response here] ## Appraisal and distribution ### Teaching and case materials #### Compare like with like Build an appraisal table using the same objectives, population and time horizon for every shortlisted option. Include access to review, meaningful correction, provider and employer burden, affordability, implementation risk and distributional effects. Record the evidence behind each judgment. Do not turn a table into a machine for producing your preferred answer. A score of four rather than three is not evidence. If you use scores, define them, make weights explicit and test whether reasonable alternative weights change the ranking. A narrative explanation should remain understandable without the arithmetic. #### Make an illustrative cost estimate Suppose a review pilot covers 2,000 applications, with a 10% review request rate and 30 minutes of reviewer time per request. That is 200 reviews and 100 reviewer hours. At an illustrative fully loaded cost of £40 per hour, direct review labour is £4,000. These figures are fictional assumptions, not a forecast or a market quote. Add training, administration, system access, evaluation and applicant effort before discussing total cost. Test demand and handling time. At 20% demand and 45 minutes per review, labour becomes 300 hours, or £12,000 using the same assumption. A credible brief explains that sensitivity instead of reporting £4,000 as a precise budget. Some outcomes are difficult to monetise; describe them clearly rather than omitting them or fabricating a price. #### Look beneath the average Who benefits from a digital review form? People confident online may use it easily; others may need supported access. Who bears compliance costs? A small employer may face a different burden from a large one. A service can have a positive average outcome while worsening access for a group already disadvantaged. Map benefits and burdens across affected groups, then examine mitigation and remaining trade-offs. Avoid treating aggregate efficiency as an automatic answer to rights or legitimacy questions. State which minimum safeguards constrain the choice and where decision-makers must make a value judgment. #### Recommend with conditions You may conclude that a funding-linked review pilot is worth testing but not ready for network-wide implementation. Specify the conditions under which it would become preferable: confirmed authority, access to records, enough reviewer capacity and evidence that affected applicants can use it. An appraisal should narrow the decision honestly, not hide uncertainty behind a single number. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Separate cash cost, social effects and a ranking For the fictional twelve-month review pilot, add £2,400 training and setup, £1,600 administration and £2,000 evaluation to direct review labour. Low-demand incremental council cost is £10,000; the stressed case is £18,000. At 200 or 400 reviews these are £50 and £45 per request respectively. Neither is cost per corrected decision: that outcome is unknown. Applicants also spend time seeking review, employers provide records, and some effects cannot be priced credibly here. These remain part of the appraisal even where no council invoice appears. An affordable budget does not establish social value. Conversely, a potentially beneficial option may be unaffordable without new funding. Suppose a digital form reaches 70 of 80 confident online users but ten of 20 people needing assistance. Access is 87.5% versus 50%; the overall 80% hides the gap. Assisted phone access is a design to test, not an assumed cure. #### Connect the reading to your practice The Green Book explicitly includes distributional effects in appraisal; the Data and AI Ethics Framework adds a fairness lens. Keep aggregate service benefits separate from who experiences errors, delay or exclusion. Explain how this changes your preferred option. Sources for this comparison: [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026); [UK Government: Data and AI Ethics Framework](https://www.gov.uk/government/publications/data-ethics-framework/data-and-ai-ethics-framework). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Compare your three options against five criteria. Calculate the two illustrative labour estimates, identify one distributional concern and test whether a changed assumption could reverse your recommendation. Show labour and the three extra cost categories, cost per request and both access rates. Explain one decision that changes if the stressed budget applies. Do not claim a cost per successful correction without that outcome. ### Review criteria - Uses consistent criteria - Calculates £4,000 and £12,000 with assumptions visible - Includes whole-service costs and unpriced effects - Tests distribution and sensitivity - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026) — Read chapter 7 on distributional analysis. Describe which applicants, employers and service teams bear the costs and benefits of each option. - [UK Government: Data and AI Ethics Framework](https://www.gov.uk/government/publications/data-ethics-framework/data-and-ai-ethics-framework) — Read Fairness and Trade-offs. Identify an impact that an average performance measure would conceal, and propose evidence to investigate it. ### Your answer [Write your response here] ## Delivery and accountability ### Teaching and case materials #### Design the work behind the announcement A recommendation is incomplete until someone can deliver it. For Northbridge's pilot, the announcement is the least demanding part. Teams need authority confirmed, agreements amended, review procedures designed, staff trained, records accessible and applicants informed. Start with the service journey. An applicant receives a decision, understands the notice, requests review, gets acknowledgment, has their case examined, receives a reasoned response and can escalate an unresolved issue. At each point ask who acts, what information they need, what deadline applies and what happens when the process fails. #### Sequence dependencies Do not schedule training before the review procedure is agreed or invite applicants into a route without staffed capacity. A useful sequence is discovery and authority check; co-design and evidence access; small operational test; pilot launch; monitored review; expansion decision. Each stage should have an exit condition rather than a date alone. For example, move from operational test to launch only when reviewers can access sample records, staff can handle the expected volume, notices have been tested with applicants and an escalation route works. These are proposed learning-case gates, not a universal standard. The team should record who checks them and who can authorise an exception. #### Give accountability a name Choose an accountable sponsor, a delivery lead, provider contacts, an evaluation lead and an owner of the review route. “The partnership” is too vague when a case needs action at 4 pm on a Friday. Specify who can pause the pilot and which issues must go to the governing board. Plan resources beyond launch: staff cover, training refreshes, maintenance, monitoring, applicant support and contract management. If these costs are unfunded, the policy is less deliverable than the announcement suggests. Add a dependency register showing the consequences of late legal advice, unavailable records or provider withdrawal. #### Treat escalation as part of the service Set out response levels for routine delay, repeated failures and serious incidents. Ensure affected applicants are supported during a pause rather than merely turning off a dashboard. Accountability includes a route to correction and learning. It is not fulfilled by assigning a senior person's name to a document that nobody uses. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Resolve the Friday-afternoon dependency The board's decision is due in week six. Records can be negotiated in weeks two to four, but renewal notice is twelve weeks and pilot staffing is not yet funded. At 4 pm on Friday, a provider cannot retrieve the inputs behind a contested rejection. The public notice promises review within five working days. A timeline must distinguish discovery milestones from permission to operate. A functioning fallback needs a named provider reviewer with source material and escalation to someone able to act; “switch to manual” alone does not create access to missing facts. If records cannot support reconsideration, communicate the limitation and use the agreed escalation route instead of sending a fictitious reasoned reply. #### Connect the reading to your practice The AI Playbook addresses operating an AI-enabled service; NIST GOVERN addresses organisational responsibility. Translate both into named owners, resources and escalation routes. A control that nobody has time or authority to exercise is not a workable delivery plan. Sources for this comparison: [UK Government: AI Playbook](https://www.gov.uk/government/publications/ai-playbook-for-the-uk-government/artificial-intelligence-playbook-for-the-uk-government-html); [NIST AI Risk Management Framework: Core](https://airc.nist.gov/airmf-resources/airmf/5-sec-core/). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Write a five-stage delivery plan with an owner, dependency, exit condition and indicative date for each stage. Include a pause authority and a fallback for applicants if the review route fails. In your delivery plan, distinguish the week-six decision from later launch. For the Friday failure, specify acknowledgment, evidence retrieval, escalation and applicant support, with a named role for each. ### Review criteria - Stages follow dependencies - Exit conditions are observable - Owners and pause authority are explicit - Fallback protects service continuity - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [UK Government: AI Playbook](https://www.gov.uk/government/publications/ai-playbook-for-the-uk-government/artificial-intelligence-playbook-for-the-uk-government-html) — Read the material on AI governance and organisational capability. Identify a staffing or implementation dependency in Northbridge's recommendation. - [NIST AI Risk Management Framework: Core](https://airc.nist.gov/airmf-resources/airmf/5-sec-core/) — Read GOVERN, focusing on responsibilities and risk-management processes. Assign owners for review, escalation and changes to the service. ### Your answer [Write your response here] ## Evaluation and adaptation ### Teaching and case materials #### Build learning into delivery Monitoring tells you what is happening: review requests, response times, costs and provider participation. Evaluation investigates why, for whom and whether the intervention made a difference. A dashboard with improving numbers is not automatically evidence that the policy caused improvement. For Northbridge, ask three kinds of question. Process: were notices understandable and reviews delivered as designed? Impact: did applicants obtain better review and correction than under the alternative? Value: were benefits worth the whole-service resources and burdens? These questions call for different evidence. #### Establish a comparison Where feasible and appropriate, compare pilot services with similar services continuing the existing process, or use a phased introduction. Check whether groups differ before launch, whether cases move between services and whether other changes happen at the same time. Randomisation is one possible method, not a slogan that resolves practical and ethical design questions. If a strong causal comparison is not feasible, be explicit. Combine before-and-after measures, process evidence, interviews and a reasoned account of alternative explanations. Do not call an association a proven effect. Good evaluation can still identify a broken mechanism and guide adaptation without claiming certainty it does not have. #### Pre-agree the decision rules Define success, acceptable burdens and conditions requiring pause before seeing the results. For the exercise, propose that expansion needs demonstrated access for different applicant groups, evidence of meaningful correction, manageable costs and no unresolved serious failure. Actual thresholds should be justified by baseline evidence and accountable judgment. Add an early review for unexpected harms. Waiting until the end of a pilot can be inappropriate when a serious problem is already visible. Conversely, changing thresholds every week to keep the preferred option alive undermines learning. Record revisions and reasons. #### Close the policy loop The final report should answer an actual decision: expand, revise, extend learning or stop. Preserve useful process improvements even if the AI-related intervention is rejected. Share limitations and negative findings. A pilot that prevents an expensive ineffective rollout can be a useful result; the policy team should not measure its success solely by the number of deployments it secures. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Calculate a comparison before claiming impact | Applicants obtaining a reasoned response | Before | After | | --- | ---: | ---: | | Pilot services | 30/100 | 60/100 | | Comparison services | 40/100 | 60/100 | The pilot improved by 30 percentage points, the comparison by 20. The difference in changes is ten percentage points, not 30. This is an illustrative difference-in-differences calculation, not a causal guarantee. You would need evidence that trends would have been comparable without the pilot, consistent measures and no differential concurrent change or spillover. A regional notice campaign occurred in both groups. Additional reviewers joined only the pilot. If the evaluated intervention includes those reviewers, assess the package; if you claim a funding condition alone caused improvement, staffing is an alternative explanation. Define the intervention and estimand—the effect you are trying to estimate—before selecting a method. #### Connect the reading to your practice Use the Magenta Book to distinguish learning about an intervention from simply reporting activity. Pair this with Service Standard point 10 on success measures. Explain how Northbridge would know whether review access improved, and what evidence would trigger a change. Sources for this comparison: [HM Treasury: The Magenta Book (2026)](https://www.gov.uk/government/publications/the-magenta-book/magenta-book-central-government-guidance-on-evaluation-html); [UK Government: Impact evaluation of AI interventions](https://www.gov.uk/government/publications/the-magenta-book/guidance-on-the-impact-evaluation-of-ai-interventions-html); [Government Digital Service: Service Standard](https://www.gov.uk/service-manual/service-standard). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Draft an evaluation plan with one process, impact and value question; evidence for each; a comparator; two alternative explanations; and explicit expand/revise/stop criteria. Calculate both changes and their difference. Add two assumptions required for a causal interpretation and state whether your plan evaluates the whole package or the funding condition alone. ### Review criteria - Distinguishes monitoring and causal inference - Uses an appropriate comparison - Checks unintended effects and group differences - Connects evidence to a decision rule - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [HM Treasury: The Magenta Book (2026)](https://www.gov.uk/government/publications/the-magenta-book/magenta-book-central-government-guidance-on-evaluation-html) — Read the evaluation-design and interpretation material. Propose an outcome measure, a comparator and a limitation on the conclusion you could draw. - [UK Government: Impact evaluation of AI interventions](https://www.gov.uk/government/publications/the-magenta-book/guidance-on-the-impact-evaluation-of-ai-interventions-html) — Read section 2.1 (evaluation approach) and section 2.3 (Establishing a baseline). Check the assumptions behind the packet’s difference-in-differences calculation. This evaluates interventions; it does not certify model accuracy. - [Government Digital Service: Service Standard](https://www.gov.uk/service-manual/service-standard) — Read point 10 on defining success and publishing performance data. Decide which Northbridge measures should be shared and what context they need. ### Your answer [Write your response here] ## Write a decision brief ### Teaching and case materials #### Write for the person deciding A decision briefing is not a condensed literature review. Its reader needs to know what you are asking them to decide, why now, what options exist and what follows from choosing one. Put the requested decision in the opening paragraph. If approval is conditional, say so there. A useful structure is: decision sought; recommendation and conditions; problem and scope; options comparison; evidence and uncertainty; delivery and resource implications; risks and mitigations; next decision. Adapt it to the institution rather than treating the headings as compulsory bureaucracy. #### Use evidence with the right verbs “The sampled applicants reported difficulty obtaining review” is different from “automated screening causes unfair hiring across the region”. The first may be supported by your interviews; the second needs much stronger evidence. Match the claim to what the methods can establish. Label illustrative assumptions and keep a record of sources. Use numbers only when they help the decision. Explain their denominator and sensitivity. A short main brief can refer to a supporting annex for methods, costing assumptions and stakeholder input. Do not hide a material limitation in an annex that changes the recommendation's meaning. #### Give alternatives a fair hearing Describe why the runner-up might be preferable. If improved manual review is cheaper and easier but lacks contractual incentives, say that. If a review requirement depends on record access, do not describe that dependency as already secured. A fair comparison strengthens the recommendation because the reader can see your reasoning. Present the strongest objection and your response. For Northbridge, it might be that additional conditions will drive employers away. Answer with the evidence you have, the evidence missing, and the pilot design or alternative that limits exposure. “We have engaged stakeholders” is not a response to the substance. #### Ask for a bounded commitment Conclude with who will do what and when the chair will see the next decision. For example: approve a discovery phase within the existing research budget, authorise contract feasibility work, and return in six weeks with a costed pilot proposal. The precise commitment should match current evidence and authority. Clarity is a service to the decision-maker, not a reason to remove complexity that matters. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Repair an overclaim A draft says: “Our interviews prove screening discrimination is widespread. Approve a £4,000 review rollout now. Stakeholders agree; logs will be available.” Your notes contain six purposively recruited accounts, a direct-labour estimate excluding £6,000 setup/administration/evaluation, disagreement from small employers and unresolved supplier log terms. Revise the verbs, cost label and decision. Removing overstatement should not make the brief indecisive. “Approve discovery within the existing budget; require a costed pilot proposal” is a concrete action. Put record access and authority beside the recommendation because they may change the preferred option. The annex can hold detail, but cannot repair a misleading opening. #### Connect the reading to your practice Use Green Book appraisal reporting to organise a recommendation, and DSIT's assurance guidance to distinguish an evidence claim from a promise. A decision-maker should be able to see the comparator, uncertainty and conditions behind your advice without reading every technical annex. Sources for this comparison: [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026); [DSIT: Introduction to AI assurance](https://www.gov.uk/government/publications/introduction-to-ai-assurance/introduction-to-ai-assurance). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Write the opening 200 words of a brief. Include the decision sought, recommendation, strongest reason, important condition and runner-up option. Then remove one claim that exceeds your evidence. Replace all four overclaims in the draft. Write a 150–200 word opening with a bounded decision, the strongest alternative and a condition that would change your recommendation. ### Review criteria - Decision appears immediately - Recommendation states its conditions - Alternative is fairly represented - Claims and numbers match their evidence - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026) — Read the balanced judgement on value for money and presentation of appraisal results. Identify a material trade-off your briefing must disclose. - [DSIT: Introduction to AI assurance](https://www.gov.uk/government/publications/introduction-to-ai-assurance/introduction-to-ai-assurance) — Read the Communicate activity and assurance mechanisms. Separate established evidence, proposed tests and residual uncertainty in your brief. ### Your answer [Write your response here] ## Capstone: your policy recommendation ### Teaching and case materials #### Your final assignment Submit a 600–900 word decision briefing for Northbridge's chair, with a compact delivery and evaluation plan. Use the work you saved in earlier lessons. You are not expected to discover a single correct intervention. You are expected to make a reasoned, bounded recommendation that another person could assess and act on. The final fictional case update is mixed. Applicant interviews identify confusing notices in two sampled services, but the sample is small and excludes people who abandoned applications. Providers can usually record a rejection reason; one supplier cannot yet provide detailed screening logs. Employers support clearer criteria but some small employers worry about delay. The partnership's legal adviser says a new funding condition may be possible at renewal, subject to detailed review. Existing funding covers discovery, but no pilot staffing budget has been approved. These facts are the exercise's assumptions, not real research. They constrain your answer. You cannot promise universal record access, immediate contract changes or a funded pilot that nobody has approved. Equally, uncertainty does not prevent a useful next decision. #### Build the briefing Lead with a decision sought today. Define the problem and scope. Compare at least three options, including the credible baseline and a non-AI service improvement. Explain how your preferred option changes behaviour, how evidence supports that mechanism and what might make the alternative better. Include a short distributional assessment: whose access improves, who may be missed and who bears costs. Name the delivery owner and two critical dependencies. Give a resource range or a method for costing it; use the illustrative labour calculation only as an assumption. Set conditions for a pilot, an escalation route and a later expand/revise/stop decision. #### Review before marking complete Read your brief as a sceptical chair. Can you identify exactly what is being approved? Are observations distinguished from causal claims? Can a delivery team turn the plan into work? Could an applicant understand how the proposal affects them? Would negative evidence change your mind? The knowledge checks test concepts; the written capstone is **self-assessed**, using the rubric and worked example below. Completion records participation and passed checks, not an externally accredited qualification or expert review. Save your final work and download your portfolio from My learning. You can revisit and improve it as your judgment develops. #### Work the case **Case evidence:** the case documents, numbers and operating constraints are invented. External findings and methods are attributed separately. Keep illustrative methods examples separate from observations in the final case update. ##### Keep the final recommendation contestable Use the case documents and calculations from earlier lessons, but separate scenario assumptions from the final update. The confusion matrix is an evidence-audit exercise; it is not a verified Northbridge deployment result. The cost range is a scenario needing a quote and a funding decision. The comparison table is a methods exercise, not pilot evidence already collected. Your brief should include an evidence register with three labels: observed in this fictional case, illustrative planning assumption, and still unknown. Cite at least two external sources for a specific analytical method or governance question, explaining their limits. A Green Book citation supports an appraisal approach; it cannot establish that Northbridge has contract authority or that your recommendation will succeed. Add a serious alternative conclusion: under what evidence would you prefer immediate manual improvement, or decline any pilot? This prevents “discovery then AI pilot” becoming a foregone conclusion. #### Connect the reading to your practice Bring appraisal, evaluation and independent accountability research together. Use a source to support a particular analytical choice, not as a badge of authority for the entire recommendation. Record where the source's setting or assumptions differ from Northbridge. Sources for this comparison: [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026); [HM Treasury: The Magenta Book (2026)](https://www.gov.uk/government/publications/the-magenta-book/magenta-book-central-government-guidance-on-evaluation-html); [Ada Lovelace Institute, AI Now and OGP: Algorithmic accountability for the public sector (2021)](https://www.adalovelaceinstitute.org/report/algorithmic-accountability-public-sector/). The application to this fictional case is AIPEX's teaching synthesis; it is not an endorsement by the source authors. ### Exercise Write your 600–900 word final briefing using the updated case. Include a three-option comparison, recommendation, authority and evidence limits, distribution, delivery dependencies, resources and evaluation decision. Minimum 350 words to submit; aim for the full brief. Use the rubric to review your answer before completing. Within your 600–900 words, include the three evidence-status labels, a strongest alternative, a reversal condition and two attributed external methods. Do not present the earlier trial arithmetic as completed local research. ### Review criteria - Seeks a specific, feasible decision now - Fairly compares three options and a non-AI alternative - Uses evidence proportionately and labels case assumptions - Addresses authority, affected groups and full costs - Assigns delivery and explicit gates - Defines evaluation and what would change the recommendation - Uses the packet accurately and separates observations, illustrative assumptions and unresolved questions ### Guided reading - [HM Treasury: The Green Book (2026)](https://www.gov.uk/government/publications/the-green-book--2/the-green-book-2026) — Use the objectives, options and appraisal material to check the recommendation against its strongest alternative. - [HM Treasury: The Magenta Book (2026)](https://www.gov.uk/government/publications/the-magenta-book/magenta-book-central-government-guidance-on-evaluation-html) — Use the theory-of-change and evaluation material to test your delivery and learning plan. - [Ada Lovelace Institute, AI Now and OGP: Algorithmic accountability for the public sector (2021)](https://www.adalovelaceinstitute.org/report/algorithmic-accountability-public-sector/) — Use one comparative example to challenge a proposed accountability mechanism. Explain its relevance and a limit on transfer to Northbridge. ### Your answer [Write your response here]