The setting
The ICO’s November 2024 report summarises consensual audits of AI recruitment developers and providers conducted August 2023–May 2024.
The decision context
A retrospective supplier-assurance exercise based on selected audits. Organisation A is the regulator’s anonymised label; it is not an invented AIPEX company or an identifiable current supplier.
The evidence to examine
Reported evidence
Scope
The audits covered sourcing, screening and selection tools. Generative AI and biometric-processing tools were excluded.
Source: Information Commissioner’s Office, AI in Recruitment Outcomes Report (2024) · Report pp. 5 and 9–10: scope and methodology ↓Reported evidence
Organisation A: test-data separation
The ICO could not establish how Organisation A kept training, testing and validation data separate in practice, or that the datasets represented relevant populations.
Source: Information Commissioner’s Office, AI in Recruitment Outcomes Report (2024) · Report p. 20: Organisation A case study ↓Reported evidence
Organisation A: monitoring and use
Organisation A relied on external adverse-impact tests using artificial data. It also inferred demographic characteristics and offered recruiters demographic filters. The ICO challenged the fairness and basis of that processing.
Source: Information Commissioner’s Office, AI in Recruitment Outcomes Report (2024) · Report pp. 24–25: Organisation A case study ↓AIPEX analysis
AIPEX reading question
A local decision still needs evidence about the particular product, version, role criteria, candidate population and review workflow.
What this does not establish
Selected consensual audits do not estimate market-wide prevalence. Historical findings do not state current law or establish whether a new supplier is suitable.
Try the decision first
Your task.
Prepare three supplier requests. Who checks each answer, and what makes a response inadequate?
- Turn each finding into an evidence request: the artefact required, the use/version it covers and the person who will examine it.
- Define an inadequate answer. A random split or a bias-free assurance statement alone does not demonstrate independence, population coverage or an effective response to failures.
- Assign a control owner, response to a failed check and a route for candidate challenge. Mark anything the public report does not establish as unresolved.
Read one defensible approach
AIPEX analysis for discussion. Other conclusions can be defensible if their evidence, assumptions and trade-offs are explicit.
- Request a versioned data-split and evaluation record, with provenance, duplicate/leakage checks and population coverage. An analyst must examine how separation works, rather than accept the existence of three dataset names.
- Request test results, denominators and the action log after failures; explain what artificial-data tests cover and leave unknown. The ICO report supports these questions, not a new local performance estimate or a universal acceptable gap.
- Record who can investigate and correct an outcome, who can suspend use and how candidates reach them. Keep data-protection and equalities decisions with the responsible specialists; the historical report is not current legal clearance.
Read the evidence / Reviewed 18 October 2026
Sources & scope.
AIPEX’s prompts and worked approaches are our own synthesis. Use the linked material to examine the method and its limits.
- Information Commissioner’s Office
AI in Recruitment Outcomes Report (2024) ↗
Report pp. 5, 9–10 (scope); p. 20 (data separation); pp. 24–25 (Organisation A and bias monitoring)
Historical findings from selected consensual audits conducted August 2023–May 2024. Generative AI was outside scope. Findings do not estimate prevalence across the recruitment market or state current law.
- Mitchell et al.
Model Cards for Model Reporting (2019) ↗
Intended use, evaluation data, metrics and subgroup performance
A reporting proposal, useful for asking what an evaluation covers and leaves out. A model card is not independent validation.
- UK Government
AI Playbook for the UK Government ↗
Human oversight; testing and evaluation; managing the AI lifecycle
Used here for operational oversight and lifecycle guidance. Its legal descriptions reflect its publication context; check current law and regulatory guidance with the responsible specialists. A completed plan is not legal clearance.