State capacity for AI: is Whitehall ready?

Executive Summary

The United Kingdom has positioned itself as a global leader in AI governance. The 2023 AI Safety Summit at Bletchley Park, the establishment of the AI Safety Institute (AISI)[2], and the “pro-innovation” regulatory framework set out by the Department for Science, Innovation and Technology (DSIT)[1]have together created a credible claim to international leadership. But governance is only half the equation. The harder question – and the one this brief addresses – is whether the British state has the internal capacity to actually use AI effectively in its own operations: to procure it wisely, deploy it safely, evaluate it rigorously, and integrate it into the fabric of public service delivery.

Our assessment, based on structured interviews with 47 senior civil servants across 14 departments[3], analysis of 63 government AI procurement exercises between 2021 and 2025, and a departmental readiness survey conducted in partnership with the Central Digital and Data Office (CDDO)[4], reveals a significant and widening gap between ambition and capability. The government’s AI rhetoric has outpaced its institutional readiness by a considerable margin. Most departments lack the technical talent to evaluate vendor claims, the procurement frameworks to buy AI systems sensibly, and the organisational culture to integrate AI into decision-making rather than treating it as a bolt-on technology project.

This brief introduces the State AI Readiness Index, an original framework for assessing departmental preparedness across five dimensions: technical talent, procurement capability, data infrastructure, leadership understanding, and organisational culture.[5]Applying this framework reveals that only three of the 14 departments assessed – HMRC, the Government Digital Service within the Cabinet Office, and the Ministry of Defence – score above the threshold we define as “deployment ready.” The remaining eleven, including departments with enormous AI exposure such as the Department for Work and Pensions (DWP) and the Home Office, fall significantly short.

The consequences of this capacity gap are not hypothetical. They are visible today in procurement exercises that take 26 months from need identification to deployment[10], in vendor contracts that lock departments into proprietary systems with no knowledge transfer, in pilot projects that never scale, and in a growing asymmetry of expertise between the public sector and the technology companies it must regulate. If the UK is serious about being an AI leader, it must invest not only in frontier research and safety science but in the unglamorous work of building state capacity: hiring technical talent, reforming procurement, and changing the culture of Whitehall.

This brief draws on the experience of the Government Digital Service (GDS) – the most successful example of embedding technical capability within UK government[24]– to argue that AI requires a similar but distinct institutional model. Where GDS was primarily about building digital services, AI adoption is fundamentally about augmenting human decision-making. This distinction matters for institutional design, talent strategy, and the relationship between central and departmental capability.

1. The capability gap

The fundamental challenge facing Whitehall is not a lack of interest in AI. Ministerial enthusiasm is abundant; every department we surveyed had at least one AI initiative either in progress or planned.[3]The problem is that most departments lack the in-house expertise to be intelligent customers of AI technology. They cannot reliably evaluate vendor claims about what AI systems can do, specify requirements that reflect operational reality, manage implementation in ways that ensure knowledge transfer, or assess whether deployed systems are actually delivering the outcomes promised. This creates what we term the “vendor trap”[9]: departments without sufficient technical expertise become captive to vendor narratives about what AI can and should do, leading systematically to over-promising and under-delivering.

The scale of the capability gap is stark. Across the 14 departments we assessed, there are approximately 740 civil servants with specialist AI or machine learning skills[8]– defined as the ability to evaluate model architectures, assess training data quality, audit algorithmic outputs, or manage AI system lifecycles. Our estimate of the minimum number required to give government baseline competence as an AI consumer and deployer is 2,500.[5] This figure does not assume government should build its own frontier models; it reflects the minimum needed to procure, deploy, oversee, and evaluate AI systems being offered by the private sector. The gap of roughly 1,760 specialists represents not just a hiring challenge but a structural vulnerability.

The distribution of existing capability is highly uneven. HMRC, which has a long history of data analytics and has invested significantly in its data science function since 2018[6], employs approximately 210 specialists with AI-relevant skills. The Ministry of Defence, through Defence Digital and the Defence AI Centre[7], accounts for another 140. GDS and CDDO together contribute roughly 95. This means that three entities account for 60 per cent of the government’s total AI capability, leaving eleven other major departments to share the remaining 295 specialists.[8]DWP, which administers Universal Credit and processes millions of decisions per year – many of which are prime candidates for AI augmentation – has fewer than 45 staff with specialist AI skills. The Home Office, responsible for border control, immigration casework, and policing technology, has approximately 55.

The capability gap has a compounding quality. Departments that lack technical expertise make poor procurement decisions, which leads to failed or underperforming projects, which in turn breeds organisational scepticism about AI, which makes it harder to secure funding and leadership support for capability building. Several interviewees described this as the “AI credibility cycle” – a pattern in which early failures, often caused by inadequate specification or vendor over-promising, create institutional resistance to further AI adoption.[3] Breaking this cycle requires upfront investment in capability before procurement, not after.

The vendor trap manifests in predictable ways. Without in-house technical expertise, departments rely on vendor-supplied assessments of what AI can achieve. These assessments are, unsurprisingly, optimistic.[9]A senior civil servant in the Home Office described a pattern we heard repeatedly across departments: “The vendor comes in with a demonstration that looks extraordinary. Nobody in the room can ask the hard questions about training data, edge cases, or how the system performs when conditions change. We sign a contract based on a demo, and then spend two years discovering the limitations.”[3]This dynamic is not unique to government, but the consequences in a public sector context – where AI systems may affect benefits decisions, immigration outcomes, or criminal justice – are considerably more serious.

2. The procurement problem

Government procurement frameworks were designed for buying defined products and services – things with stable specifications, predictable performance characteristics, and clear acceptance criteria. AI systems are none of these things. They require iterative development, ongoing refinement based on deployment context, continuous evaluation as both capabilities and operational conditions change, and a fundamentally different relationship between buyer and supplier than traditional outsourcing models assume.[10] The mismatch between existing procurement machinery and the nature of AI technology is not a minor friction; it is a structural barrier to effective government AI adoption.

Our analysis of 63 government AI procurement exercises conducted between 2021 and 2025 reveals that the average time from identification of need to operational deployment is 26 months.[13]This figure includes the full cycle: business case development (typically 4–6 months), procurement process (8–12 months), contracting (2–3 months), implementation (4–6 months), and acceptance testing (2–3 months). By comparison, equivalent AI deployments in FTSE 250 companies average 6–9 months from decision to deployment. The gap is not primarily attributable to appropriate democratic caution or accountability requirements. Our analysis suggests that only about 4 months of the difference is explained by governance and oversight requirements that have no private-sector equivalent. The remaining 13–16 months of additional time reflects procurement processes that are simply not designed for the technology they are trying to acquire.

The Crown Commercial Service (CCS) frameworks through which most government AI procurement is channelled – principally the G-Cloud and Digital Outcomes and Specialists frameworks[11]– were designed for cloud hosting and digital service delivery. They do not accommodate the iterative, experimental nature of AI deployment. Specification requirements assume that the buyer knows what they want before procurement begins, which is rarely true with AI. Evaluation criteria weight price and compliance over technical capability and domain understanding. Contract structures assume a defined deliverable rather than an ongoing capability. The result is that procurement exercises systematically select for vendors who are good at writing bids rather than vendors who are good at deploying AI.

A particularly damaging feature of current procurement practice is the treatment of AI as a capital expenditure rather than an ongoing capability investment.[12]Departmental finance teams, operating within HM Treasury spending review cycles, want AI projects to have defined costs, timelines, and deliverables. This framing is at odds with the reality that AI systems require continuous monitoring, retraining, and adaptation. The result is projects that are funded for deployment but not for the ongoing investment needed to keep them effective – leading to a pattern of “deploy and decay” that several interviewees identified as a growing problem.[3]

Intellectual property and data ownership provisions in standard government contracts are also poorly suited to AI. In 38 of the 63 procurement exercises we reviewed, the resulting contracts gave vendors ownership of models trained on government data, with the department receiving only a licence to use the outputs.[13]This means that government is effectively subsidising the development of commercial AI products using public data, while retaining neither the models nor the institutional knowledge to maintain or modify them. When contracts expire, departments face a choice between renewal with the incumbent – often at significantly increased cost – or starting from scratch with a new vendor. This lock-in dynamic is well understood in IT outsourcing generally, but the asymmetry of expertise in AI makes it particularly acute.

There are emerging examples of better practice. HMRC’s approach to its fraud detection AI programme involved a hybrid model in which external vendors provided initial development support while an internal team was embedded throughout, progressively taking over model maintenance and refinement.[14]The MOD’s Defence AI Centre has experimented with “challenge-based” procurement that defines problems rather than solutions, allowing vendors to propose approaches that are then evaluated through practical demonstration rather than written bids.[15] These examples suggest that reform is possible within existing legal frameworks; what is needed is institutional permission and technical capability to do procurement differently.

3. The talent pipeline

The civil service cannot compete with the private sector on AI talent compensation. This is widely acknowledged.[16]A mid-career machine learning engineer in the private sector can expect total compensation of £120,000–£180,000 in London; the equivalent civil service role, typically graded at SEO or Grade 7, offers £45,000–£68,000. At senior levels the gap is even more pronounced: a principal data scientist at a major technology company earns £200,000–£350,000, while a Grade 6 or SCS1 technical leader in government earns £75,000–£95,000.[20] The Digital, Data and Technology (DDaT) pay framework, introduced to address this gap, has helped at junior levels but remains uncompetitive for the senior technical leaders who set architectural direction, evaluate vendor proposals, and make deployment decisions.

But compensation is only part of the problem, and arguably not even the most important part. Our interviews with both serving and former government technologists consistently identified three non-compensation factors that are at least as significant in deterring AI talent from public service.[19] First, career structures that force technologists into general management to advance. The Senior Civil Service remains overwhelmingly generalist in its culture and promotion criteria; there is no credible technical career track that leads to the most senior grades without progressively abandoning technical work.[21] Second, working practices that frustrate technical staff: security-cleared laptops with restrictive software policies, inability to use modern development tools, and IT environments that are years behind industry standard.[17]Third, organisational cultures that treat technical staff as service providers rather than decision-making partners. Multiple interviewees described being brought in to “implement the solution” after policy decisions had already been made, rather than being involved in problem definition.[3]

The Civil Service Commission’s own data illustrates the retention challenge. Of the 180 specialist AI and data science roles filled through external recruitment between 2022 and 2024, 67 – 37 per cent – left within 18 months.[18]Exit interviews, where conducted, cite a consistent set of factors: frustration with the pace of decision-making, inability to use the tools and methods they consider standard practice, and a perception that technical expertise is undervalued relative to policy and management skills. The cost of this turnover extends beyond recruitment expenses; each departure takes institutional knowledge and, in a small specialist community, damages the government’s reputation as an employer.

Government does not need to match private sector salaries for every technical role. The mission-driven nature of public service is a genuine attractor for many technologists, and our interviews confirm that a significant cohort of AI specialists would accept a salary discount of 20–30 per cent to work on problems of public significance.[19]But this discount has limits. The current gap of 50–70 per cent at senior levels exceeds what mission motivation can bridge. Moreover, government needs to offer competitive compensation for a critical mass of senior technical leaders – perhaps 200–300 individuals across Whitehall – who can evaluate AI systems, challenge vendor claims, set technical standards, and mentor junior staff. These are the roles where the compensation gap does the most damage, because these individuals set the capability ceiling for their entire department.

The pipeline problem is compounded by geography and security clearance. Many AI roles in government require Developed Vetting (DV) or Security Check (SC) clearance, which adds 3–9 months to the recruitment process and excludes non-UK nationals who constitute a significant proportion of the AI talent pool.[16]The concentration of senior government roles in London, combined with a cost of living that makes civil service salaries even less competitive in real terms, further narrows the available talent pool. Remote and hybrid working policies have helped, but the most sensitive AI work – precisely the work where government capability matters most – often requires on-site presence in secure facilities.

4. Institutional culture

The cultural barriers to AI adoption in Whitehall are at least as significant as the technical and procurement barriers, and considerably harder to address through policy intervention. The British civil service has a deeply embedded generalist culture, rooted in the Northcote–Trevelyan reforms of 1854 and reinforced by 170 years of institutional practice.[21] The ideal civil servant, in this tradition, is a gifted generalist who can move between departments and policy areas, bringing judgement and analytical rigour rather than specialist technical knowledge. This model has many virtues, but it creates an institutional environment in which technical expertise is structurally undervalued and technologists are seen as support staff rather than strategic advisors.

This cultural orientation manifests in how AI decisions are made. In the majority of departments we assessed, decisions about AI adoption – which systems to procure, where to deploy them, what problems to prioritise – are made by policy officials and senior leaders with little or no technical background.[3] Technical staff are consulted, but typically late in the process and in an advisory rather than decision-making capacity. The result is that AI strategy is shaped by people who understand policy objectives but not technical constraints, leading to unrealistic expectations, poorly specified requirements, and a disconnect between what is promised and what is deliverable.

Risk aversion compounds the cultural challenge. The civil service operates in an environment of intense public scrutiny, where failures are investigated by select committees and reported in the media, while successes are invisible.[22]This asymmetry creates a rational institutional preference for inaction over experimentation. AI adoption inherently involves uncertainty – systems may not perform as expected, edge cases may produce errors, and the consequences of those errors in a government context can be severe. The rational response for a risk-averse institution is to demand certainty before deployment, which is fundamentally incompatible with the iterative, experimental approach that effective AI adoption requires. Several interviewees described a pattern of “analysis paralysis” in which AI projects remain permanently in pilot stage because nobody is willing to take the decision to scale.[3]

The rotation system, in which civil servants move between roles every two to three years, creates a further obstacle. AI programmes typically require sustained leadership over periods longer than a standard posting. The senior responsible owner (SRO) who commissions an AI system may have moved on before it is deployed, and their successor may have different priorities or less understanding of the technical decisions already made. This lack of continuity disrupts long-term capability building and makes it difficult to maintain institutional knowledge about AI systems and their performance.[16]Three interviewees specifically cited SRO turnover as the primary reason their AI programmes had stalled or been de-prioritised.[3]

There are, however, pockets of cultural innovation. HMRC’s decision to co-locate data scientists with policy and operational teams, giving them equal status in project governance, has demonstrably improved the quality of AI deployment in tax compliance.[14]The NHS’s AI Lab, while not without its challenges, has created a model in which clinicians and technologists work as genuine partners.[23] These examples suggest that cultural change is possible, but it requires deliberate institutional design and sustained leadership commitment. It cannot be achieved through memoranda or training programmes alone.

5. What GDS teaches us

The Government Digital Service, established in 2011, remains the most important precedent for embedding technical capability within the British state.[24]Before GDS, government digital services were largely outsourced to major systems integrators under contracts that were expensive, inflexible, and produced services that were difficult for citizens to use. GDS demonstrated that a central technical team, empowered by ministerial support and armed with clear standards, could transform government service delivery. The GOV.UK platform, the Digital Service Standard, and the “digital by default” agenda showed that government could be a sophisticated technology organisation when it chose to invest in internal capability.

The GDS model offers several lessons for AI adoption. First, central capability matters. GDS succeeded because it was a central team with the authority to set standards and the expertise to help departments meet them. Without a central body that combines technical credibility with institutional authority, individual departments will continue to make inconsistent, often poor decisions about AI. CDDO, which succeeded GDS as the centre of government digital capability, has some of this mandate but has been significantly less well resourced and has not been given the same level of ministerial backing.[4] Second, standards drive behaviour. The Digital Service Standard gave GDS a lever to influence departmental practice that was more effective than any number of strategy documents.[17]An equivalent AI deployment standard – defining minimum requirements for in-house expertise, data quality, evaluation methodology, and ongoing monitoring – would be a powerful tool for raising the floor of government AI capability.

But AI is not digital, and the GDS model cannot simply be replicated. The critical distinction is that GDS was primarily about building services – transactional systems through which citizens interact with government. The design challenge was principally one of user experience, technical architecture, and agile delivery. AI adoption is fundamentally different because it is not primarily about building new services but about augmenting human decision-making within existing operational processes. When DWP considers using AI to assist with benefit eligibility assessments, or the Home Office explores AI-assisted visa processing, the challenge is not to build a new digital service but to integrate an AI system into an existing decision-making workflow in a way that improves outcomes without undermining accountability, fairness, or public trust.

This distinction has profound implications for institutional design. GDS could operate effectively as a central team because digital service design is relatively portable across departments – the principles of good user experience and agile development apply regardless of whether you are building a tax service or a passport application. AI deployment is far more domain-specific. An AI system for tax fraud detection requires deep knowledge of tax law, compliance patterns, and HMRC operational practice. An AI system for immigration casework requires understanding of immigration rules, case officer workflows, and the specific risks of algorithmic decision-making in a rights-affecting context. A central AI team cannot provide this domain expertise; it must be developed within departments, with central support for standards, methodology, and shared infrastructure.

The GDS experience also offers a cautionary lesson about sustainability. GDS’s influence peaked between 2013 and 2016, driven by strong ministerial support and charismatic leadership. After key leaders departed and ministerial attention moved on, the organisation lost momentum and was eventually absorbed into a broader Cabinet Office function.[24] Any institutional model for AI capability must be designed for durability, not dependent on individual champions or the enthusiasm of a particular Secretary of State. This argues for a statutory or at minimum a formal framework basis for an AI capability function, with protected funding and clear accountability to Parliament rather than solely to ministers.

The most transferable lesson from GDS is perhaps the most counterintuitive: that the best way to build departmental capability is to start by demonstrating what good looks like. GDS did not begin by writing strategy documents or commissioning capability reviews. It began by building GOV.UK – a working product that showed, concretely, what government digital services could be. An AI capability function should similarly start with demonstration projects: working AI deployments, developed in partnership with departments, that show what responsible, effective government AI looks like in practice. These reference implementations would do more to shift institutional culture than any number of frameworks and guidelines.

6. International benchmarks

The UK’s capacity challenges are not unique, but several comparable governments have made more progress in addressing them.[25]Examining international approaches reveals both models to learn from and cautionary tales. The most instructive comparisons are with countries of similar governmental complexity and democratic accountability requirements: the United States, Canada, Singapore, Estonia, and France.

The United States, through the Office of Management and Budget’s AI governance directives and the establishment of Chief AI Officers in every federal agency, has created a distributed accountability model that the UK lacks. Each major federal department now has a named senior official responsible for AI adoption and risk management, with a reporting line to the agency head and a dotted line to the White House Office of Science and Technology Policy. The US Digital Service and 18F have also pioneered models for embedding technical talent in government on time-limited “tours of duty” – typically two-year assignments that bring private-sector expertise into government without requiring permanent career transitions. As of early 2026, approximately 1,200 AI specialists work across federal agencies, a figure roughly proportional to the UK’s gap when adjusted for the size of the federal workforce.[25]

Singapore offers perhaps the most relevant model for a small, centralised government seeking to build AI capability rapidly. The Government Technology Agency (GovTech) employs over 400 AI and data science specialists and operates as an internal technology provider to all government ministries.[26]GovTech’s compensation packages are explicitly benchmarked to the private sector at approximately 80 per cent of equivalent roles, and it offers a technical career track that extends to the most senior grades without requiring a transition to management. Singapore’s AI governance framework, developed by the Infocomm Media Development Authority, is integrated directly into GovTech’s deployment processes rather than existing as a separate compliance exercise. The result is an AI adoption rate across government that significantly exceeds the UK’s, with notably fewer high-profile failures.

Estonia, often cited as a digital government exemplar, provides a different kind of lesson. Its success in digital services rests on foundational investments in data infrastructure – particularly the X-Road data exchange layer – that enable AI applications to access and integrate data across government without the departmental silos that plague UK government data architecture.[27]Estonia’s Krätt initiative, which established a legal and operational framework for government AI use, addressed governance questions early rather than treating them as an afterthought. The UK’s data infrastructure is significantly more fragmented, with departments operating incompatible systems and applying inconsistent data standards, creating a practical barrier to AI deployment that no amount of procurement reform can overcome.

France’s approach under the national AI strategy, led by INRIA and the Direction Interministérielle du Numérique, has focused heavily on building sovereign AI capability within the public sector.[29]The French government has invested directly in training programmes that produce AI specialists for government service, with bursary schemes that exchange tuition support for committed periods of public service. Canada’s AI-focused procurement reforms through its Treasury Board Secretariat have reduced average AI procurement timelines to approximately 14 months – still longer than the private sector but significantly faster than the UK’s 26-month average.[28]The Canadian approach of “agile procurement” for AI, which allows contracts to be awarded on the basis of working prototypes rather than written bids, offers a practical model for CCS reform.

7. Policy recommendations

Our recommendations focus on building lasting institutional capability rather than one-off investments. They are sequenced to reflect both urgency and dependency: some measures can and should be implemented immediately within existing frameworks, while others require legislative or spending review action. Together, they constitute a comprehensive programme for building the state capacity that the UK’s AI ambitions require.

First, establish a permanent AI Delivery Unit within the Cabinet Office, reporting to the Chancellor of the Duchy of Lancaster, with an initial complement of 150 technical specialists and a mandate to embed teams in departments. This unit should combine the standard- setting authority of CDDO with the delivery capability of the original GDS, specifically adapted for AI.[24]It should have the power to conduct AI deployment assessments – analogous to the old GDS service assessments – and to require remediation before departments proceed with AI deployments that do not meet baseline standards. The unit should also maintain a shared AI infrastructure platform providing common tools, evaluation frameworks, and pre-assessed vendor components that departments can draw on rather than procuring independently.

Second, reform Crown Commercial Service procurement frameworks to enable iterative AI acquisition. This means creating an AI-specific procurement pathway with three key features: challenge-based specification (defining problems, not solutions), prototype-based evaluation (assessing working demonstrations, not written proposals), and 90-day review cycles (allowing contracts to be adjusted or terminated based on actual performance rather than predicted outcomes).[11]The target should be to reduce average AI procurement timelines from 26 months to 12 months within three years, with a long-term target of 9 months. CCS should also mandate that all AI contracts above £1 million include provisions for model ownership or model access, knowledge transfer plans, and exit strategies that prevent vendor lock-in.

Third, launch a Civil Service AI Fellowship programme, administered by the Civil Service Commission in partnership with DSIT, offering salaries benchmarked at 80 per cent of private-sector equivalents for up to 300 senior technical leaders across government.[26]Fellows should be appointed on three-to-five-year terms, exempted from the standard rotation system, and given dual reporting lines to both their departmental permanent secretary and the head of the AI Delivery Unit. The fellowship should also include a “tour of duty” track for private-sector specialists willing to spend 18–24 months in government, modelled on the US Digital Service approach. The target should be to reach 2,500 specialist AI roles across government by 2029, approximately tripling current capacity.

Fourth, mandate that all departments with significant AI exposure – defined as any department deploying AI systems that affect decisions about individuals or allocate public resources – maintain a minimum ratio of one in-house AI specialist for every three external contractor staff on AI projects.[10]This ratio requirement, enforced through the AI Delivery Unit’s assessment process, would ensure meaningful knowledge transfer and reduce the vendor dependency that currently characterises most government AI programmes. Departments that cannot meet this ratio should be required to either invest in recruitment or reduce the scope of their AI deployment until internal capability catches up.

Fifth, invest in the data infrastructure that AI deployment requires. CDDO’s National Data Strategy should be resourced and accelerated, with a specific focus on creating interoperable data standards across departments, building shared data platforms that enable AI systems to access the information they need without duplicating datasets across departmental boundaries, and establishing clear governance frameworks for the use of linked administrative data in AI training.[4] Without this investment, AI adoption will continue to be limited by the fragmented, inconsistent, and poorly documented data landscape that characterises most of Whitehall.

The total investment required is significant but modest relative to the scale of the opportunity. We estimate an annual cost of approximately £320 million once the programme reaches full scale: £180 million for the AI Fellowship and expanded technical recruitment, £60 million for the AI Delivery Unit, £50 million for shared AI infrastructure, and £30 million for procurement reform and training.[5]This represents less than 0.03 per cent of total government expenditure and should be evaluated against the potential for AI to deliver significant efficiency gains across the £1.2 trillion public spending envelope. The alternative – continuing to muddle through with inadequate capability – is not cost-free; it means billions spent on poorly specified AI contracts, failed implementations, and a widening gap between what government promises its citizens and what it can actually deliver.