Digital Economy Dispatch #175 -- Lessons for Accelerating Delivery of AI-at-Scale

Digital Economy Dispatch #175 -- Lessons for Accelerating Delivery of AI-at-Scale
17th March 2024

AI is a hot topic. There is a lot of excitement about its potential to change just about all the products and services we use every day. However, large-scale adoption of AI solutions, what I have started to call AI-at-scale, faces challenges familiar to anyone who has been involved with digital transformation efforts over the past decade. While small-scale experiments and limited use cases abound, expanding the range, application, and resilience of those solutions is proving to be a much harder nut to crack. Obtaining sufficient high-quality data, integrating AI with existing systems, overcoming talent shortage, and managing ethical considerations are just a few of the many key hurdles faced by large established organizations (LEOs) in the public and private sector as they take on this task.

Studies, research, and case studies of AI adoption all indicate that leaders must address these issues to unlock the true potential of AI and bring the benefits of AI to all those in the organization. Where should organization’s place their focus and how do digital leaders identify the barriers to be overcome to accelerate AI adoption?

To make progress, a critical first step is to broaden our understanding of the scope and characteristics of the challenges being faced in delivering AI-at-scale by learning from those around us. To help in this task, I have been fortunate to be engaged over the past few months in 2 initiatives that shine a spotlight on the issues and provide lessons on how to accelerate AI-at-scale. Their results have now been released and make interesting reading for anyone wanting to accelerate AI adoption.

The first of these is a broad survey conducted by the Digital Leaders network into the attitudes toward adoption and use of AI of digital leaders across the public and private sectors. The second was a study carried out by the UK National Audit Office (NAO) and involved a more substantial examination of the current use of AI across UK government agencies. In both of these efforts I was a member of the team conducting the study and contributed as a co-author in producing the final report.

Digital Leaders Attitudes to AI Survey

An online survey based on questions concerning digital leader’s attitudes to AI use in their organizations was conducted in December 2023 by the Digital Leaders community and resulted in 577 completed responses. The majority of respondents (50%) were from the public sector, with the remainder split between academia (5%), charity (17%), and the private sectors (28%). What makes this survey particularly valuable is the seniority of those responding: 58% of respondents identified themselves as digital leaders at C-Suite level and 42% at the Senior Management Team level.

The results of this survey confirm widespread interest in AI from all digital leaders but also highlights the challenges they perceive in AI adoption such as the need for better data management infrastructure, the high cost of talent acquisition and development, and the lack of robust ethical frameworks for successful adoption. Reviewing the detailed responses reveals 5 key points that offer a broad snapshot of the state of AI-at-scale:

1. AI is already widely discussed. AI is a major topic among digital leaders, with most survey respondents reporting weekly discussions and interactions with AI, and over a third using it daily. This frequent engagement is driving significant debate about AI at senior leadership levels.

2. AI use is a mixed picture. While awareness of AI is high, many surveyed organizations haven't identified practical uses for it or assessed its business impact. This lack of clear strategy extends to generative AI, with most organizations lacking policies to govern its use.

3. AI adoption is causing challenges. Implementing AI faces hurdles common to digital transformations in large organizations. While ROI concerns exist (almost half unsure of positive impact), bigger issues lie in talent acquisition/retention and integrating AI into existing workflows (both cited by over half as significant barriers). Interestingly, job loss fears were a lesser concern for most respondents (less than a quarter).

4. AI impact on systems performance is unclear. Despite interest in AI, there are concerns about its real-world use. Reliability and data privacy are major issues, with less than a quarter confident in AI for critical tasks and over 90% worried about data privacy.

5. AI brings new leadership concerns. Digital leaders prioritize building trust in AI by tackling ethics, bias, and transparency. However, the survey reveals a concerning lack of preparedness for upcoming regulations and responsible AI frameworks, with over 60% of respondents expressing worries in these areas.

Overall, the Digital Leaders AI attitudes survey confirms the high expectations being created for AI in many organizations. However, it also reinforces concerns from leaders about their ability to scale AI adoption in a responsible and appropriate way.

NAO’s “AI in Government” Study

In contrast to the Digital Leaders survey’s focus on AI attitudes, the report by the NAO released on 15^th March 2024 presents a more detailed and comprehensive review of the current state of AI adoption across the UK government based on combining insights from of a survey completed by 89 government bodies, a wide number of interviews, 4 case study descriptions, and substantial background research. The report is a “value for money” assessment submitted to parliament to monitor on-going actions on AI deployment and provide input to future policy actions.

In recent months, the UK government has highlighted the potential of AI to transform public services in the UK, emphasizing its importance in generating performance improvements and driving cost savings. Based on these aspirations, the government has been developing strategies to leverage AI and supporting government agencies to expand its use through a number of investments and incentives. In this context, the NAO “value for money” study was designed to understand approaches to AI use across the UK government to maximize the opportunities and mitigate the risks in delivering these AI benefits in providing public services.

The key finding from the study was that while some government bodies have begun implementing AI, widespread adoption is in its early stages and remains limited. The report highlights that achieving AI-at-scale requires not only technological investment, but also significant changes to internal practices, external governance processes, and workforce capabilities. Historically, meeting these needs has been found to be severely challenging in large-scale digital change programmes in UK government. The study emphasizes that applying the lessons from these experiences will be important as the UK government drives its AI-at-scale ambitions forward.

Additionally, the NAO study found that there are specific areas of concern to address if UK government is to broaden its AI adoption and meet the targets being set for AI deployment. Amongst the most challenging barriers to address, the survey carried out as part of the NAO study highlighted the need for further support to address potential legal risks, improve privacy and data protection, and defend against cyber attacks and security breaches.

Unsurprisingly given the context, the NAO study also placed a particular spotlight on the relationships that exist between the various UK government agencies with responsibility for defining, delivering, and assessing progress in AI adoption. As with any large, complex organization, the internal structures, processes, and mechanisms for governance play an important role in determining the pace at which widescale change can be carried out. In particular, the report identifies the tensions that exist between government teams focused on driving AI innovation in specific domains and the range of compliance, reporting, assessment, and governance obligations typical of all public sector activities. Achieving AI-at-scale requires finding ways to balance these competing concerns by improving communication, encouraging knowledge and asset sharing, and clarifying overlapping roles and responsibilities.

To address these challenges, the NAO report highlights the importance of robust central government support, including ensuring clear ownership of the AI strategy, aligning funding allocation efforts, and refining implementation plans to emphasize measurable goals. Furthermore, the report emphasizes the importance of tying AI adoption to core digital transformation improvements including modernizing IT infrastructure, developing a skilled workforce, and establishing clear guidelines for managing risks such as data bias and data security. By effectively addressing these considerations, the report suggests that the transformative potential of AI in public services can be brought more sharply into focus.

Taking the Next Steps in AI-at-Scale

Both of these studies draw attention to the challenges of accelerating AI-at-scale. The combined insights from the Digital Leaders AI attitudes survey and the NAO's "AI in Government" study offer valuable lessons for all digital leaders looking to accelerate responsible and impactful AI-at-scale within their organizations:

• Bridge the Gap Between Ambition and Action. While interest in AI is high, organizations that lack clear strategies for implementation will struggle to meet expectations. Leaders must prioritize identifying practical use cases with a demonstrable ROI, ensuring alignment with core business goals.

• Prioritize Talent and Infrastructure. Skilled talent and robust data infrastructure are fundamental for successful AI integration. Leaders must match ambitions to their investment in talent acquisition, development, and reskilling programs focused on expanding AI expertise. Additionally, a focus on modernizing IT infrastructure is essential to support data ingestion, storage, and analysis required for AI operations.

• Build Trust and Mitigate Risks. Being explicit about ethical considerations and data privacy concerns is paramount. Leaders must prioritize developing robust governance frameworks for AI development and deployment. This includes establishing clear lines of authority, communicating guidelines for data management, addressing potential biases in algorithms, and ensuring responsible AI use is aligned with rapidly-changing regulations.

By addressing these key lessons, digital leaders can accelerate the path towards AI-at-scale, unlock the true potential of AI, and enable their organizations to leverage this transformative technology responsibly and effectively.