With the wealth of discourse and subject matter material available, it can be difficult for ICAEW members, and the businesses that they are part of, to keep sight of the key questions that should be asked when looking to adopt, implement and scale AI. We believe that the questions in this article, which also consider the impact of the EU AI Act (“AI Act”), can help businesses gain a clearer perspective of how AI can unlock potential, disrupt their operations, and drive future growth.

Adopt

As part of your AI strategy, are you clear if and where AI might benefit your organisation?

Identifying whether the implementation of AI tools will support the needs and goals of their business should be the first question that members should ask. The increased interest generated by recent developments in AI technology presents the risk that organisations will invest in AI tools without a clear rationale out of concern of being surpassed by their competitors should they not make such an investment.

Members should identify areas and processes within their organisations where performance could be improved through the use of AI tools, for example, data analysis and the elimination of repetitive tasks. Consideration should then be given to how using AI tools in these areas would result in greater benefits than alternative solutions. In doing so, members need to be clear whether the AI system adopted meet the AI Act’s definition of AI, and how the AI system impacts the fundamental rights of customers.

How much are you prepared to invest?

Establishing a budget for adopting and implementing AI in your business will influence not only the scale and scope of the project, but the sophistication of the AI tools that are developed or purchased. Investment needs to be seen as long term, as significant returns may not emerge in the initial years of AI adoption.

In addition to the financial considerations, members will need to consider the amount of time invested by the project’s stakeholders, as the participation and cooperation of the leadership, technology, and legal teams as well as any other internal stakeholders will be crucial to the project’s success.

Complying with regulatory requirements will also be costly. Under the EU’s new regulatory regime, technical documentation must be produced, customers must know when they are interacting with an AI system and codes of conduct, transparency requirements and/ or conformity assessments will need to be generated dependent on the risk categorisation of the AI system.

Have you considered the potential risks?

Understanding the risks associated with adopting of AI is as important as determining the benefits. Any use of AI tools requires a business to assess the risk posed by the need to comply with a growing volume of AI and data handling legislation and regulations such as the AI Act or GDPR.

Compliance with legislation is not the only source of risk, unrestrained project costs, reputational damage caused by data or privacy breaches, copyright infringement and the spread of misinformation or biased output can all impact your business. Members should ensure that their organisation’s risk appetite, the scale of the AI implementation project and supporting governance arrangements are aligned, and that the requisite skills base is in place.

Implement

Should you develop AI internally or outsource?

As part of the implementation process, businesses should consider whether they have the necessary expertise and resources to develop in-house AI tools and solutions, or whether they should partner with external vendors who can provide third-party solutions. For most businesses outside of the technology sector, or for those of a smaller scale, the latter option may be more realistic given the complexity of the project and the specialist skills needed to develop AI tools.

Therefore, members considering using third-party AI solutions should instead focus on the capabilities and reputation of the third party as well as establishing a scope and timeline for the project that the third party can meet. Furthermore, the AI Act requires members to consider their business’ position in the AI chain, this being an importer, distributor, deployer or provider, and the associated the roles and responsibilities of each position (e.g. completing conformity assessments).

How will you implement AI in your business?

A successful implementation of AI tools within your business relies upon three key factors, project management, the data that will be used and the governance and compliance process.

The AI implementation project should have a defined project leader who can manage the project, act as point of contact for other teams in the organisation and who is accountable to the board for its success. Once implemented, firms will need to allocate responsibility for ensuring that the AI Act’s transparency and conformity assessment requirements are met.

The data used to inform the AI tools directly correlates to the quality of results generated. To ensure that the insights produced are accurate and free from bias, members should carefully consider the quantity, quality and variety of datasets used to train AI models, which can be an expensive and time-consuming process, that may contravene sensitive data requirements if done incorrectly. For example, multiple data sets, used in isolation, may mean individuals cannot be identified, but if used in conjunction with other data sets, may begin to.

For AI systems impacted by the EU AI Act, the extent of the governance and compliance process will depend on whether the system is categorised as “High Risk”. Not all AI systems will be listed in the legislation and firms will need to consider which category their system falls into where the answer is not clear cut.

What about your people?

Gaining the support of your people is one of the most important steps in implementing AI into your business. Aside from dispelling concerns that the adoption of AI could threaten their jobs, a successful AI implementation project requires staff engagement and training. Under the AI Act, a risk management system that takes into account potential risks of the AI system and potential for over-reliability on the system, must be in place; use case domain knowledge, not just AI knowledge, will be essential.

You need to demonstrate the benefits of AI to your teams through clear communication about the aims of the AI implementation project and clear use cases demonstrating how they will be working with the new AI systems as well as the tangible benefits that they will experience.

Businesses should consider how they will upskill their employees to enable them to effectively utilise AI tools being implemented. A further consideration for members is how the AI Act’s requirement for human oversight of AI systems will be implemented. That oversight must understand the capacity and limitations of the AI system, remain aware of the tendency to over-rely on outputs provided by AI systems used by humans to make decisions and be able to intervene by means of a “stop” button. If an AI system is taken off the market, deployers, importers and users must be made aware.

Scale

Have you considered the long-term vision for AI in your business?

Developing a long-term vision for AI is a vital next step in scaling its usage in your business. Along with a consideration of how your industry will be disrupted by AI, regardless of whether your business’ initial AI adoption was a pilot transformation of a single team, or an entire department, the results of the project will be invaluable in shaping a longer-term vision for AI usage within your business.

Given the rapid development of AI tools and solutions, developing a proactive and forward-looking AI strategy will ensure that your business remains informed of emerging AI trends, developments and regulations and that your organisation’s use of AI evolves to promote future growth. Horizon scanning tools can be effectively used to monitor developments, particularly for regulatory rules and guidelines that come from different continents and bodies, which can be complex and overlap. In the UK, for example, firms will need to consider how their AI system delivers requirements under the Consumer Duty.

How will AI adoption impact your stakeholders?

How your business adopts AI will influence your relationships with many stakeholder groups. What shape the impact on stakeholders such as clients, shareholders, suppliers, and regulators will vary depending upon your industry, as well as the scale and risk profile of the AI implementation project.

Understanding and managing the impact of AI upon your stakeholders, as well as their perception of the technology, is an important step in scaling the use of AI throughout your business. For example, shareholders may be enthusiastic about the proactive market signals AI implementation sends, whereas customers may have concerns about how their data will be used and protected. Successfully balancing these concerns will influence the extent to which AI can be scaled in your business and the success of doing so.

How will your business’ structure and financial model change as you scale AI?

As your business begins to scale AI, bringing changes to products, ways of working, roles, and stakeholders, it is inevitable that structure of the business will have to be reorganised to accommodate these changes. Such changes could include new roles such as those around AI governance and compliance, business units to analyse and monitor the data used and generated by AI, reporting lines or even entire divisions devoted to new products or services arising from AI adoption.

Members should also consider how their business’ financial model will change as a result of scaling AI. Business will have to assess how the costs associated with streamlining processes, compliance with legislation such as the AI Act, additional server usage, data protection, and others will impact profitability, or will be offset by additional revenue streams generated by new products and services arising from AI adoption. Furthermore, depending on their scale, businesses will have to consider the perception of external finance providers in how their AI scaling plans are achieved.

Conclusion

Although the task of implementing and scaling AI within your business poses challenges and uncertainties, it is notable that implementing AI is not new and the potential benefits for members and their businesses are significant.

Members should approach AI adoption with clear goals and a defined plan, keeping sight of the costs, risks, the need for oversight and governance to comply with legislation such as the AI Act, as well as data security and privacy laws. There is also a renewed need for increased rigour around documentation and transparency.

Risk management systems will need to take account of foreseeable risks associated with your AI systems and human oversight remains key. The specific roles that provide that human oversight is worth spending time on understanding, including who is using your AI and what information they require. As a member of the AI chain, it is also important to know your responsibility compared to that of other actors in the AI lifecycle.

The successful participation and engagement of internal and external stakeholders in this process is critical to your success.

Finally, members should have a long term vision for the future of AI in their business and be open to the transformative influence that AI can, and indeed will have on their business.