Large language models (LLMs) that underpin Generative AI (GenAI) tools require high-quality data from trusted sources for each specific use case. Organisations using these LLMs to create and configure GenAI-enabled tools must first understand the models and know which is the right model for a particular use case.

Some models are very good at summarising, others are good at search. Attention to the safety and ethical implications of AI are of the utmost importance, and that depends on the quality of the data that goes into training the LLMs.

At PwC, for example, the firm has created an AI tool for all UK tax legislation and case law, which the firm calls a ‘UK tax assistant’, which is available to all 2,300 PwC UK Tax professionals. This is based on technology from OpenAI and Harvey.

In most cases, when businesses refer to ‘training’ AI models, what they are actually doing is using AI tools at one of three levels, according to Tamsin Tinsley, Technology, Data and Third Party Risk Management Leader at PwC.

The first level, she explains, refers to asking questions, otherwise called prompt engineering, which in time experts say will become an art and a science. The next level is inputting specific data into an AI tool, such as documents and reports, and then interrogating the tool for a specific outcome, such as summarising a report.

The final and third level refers to creating a RAG (Retrieval-Augmented Generation) or database to generate a more accurate response. The database might include all UK audit regulations, for example, which can be used by the whole organisation.

Tinsley says: “Once you have that database you can use the model to interrogate it, using that database to answer specific questions. Broadly speaking when businesses talk about training AI models, that's what they're talking about.”

The difference between these bespoke AI tools and the likes of ChatGPT is that many organisations are creating them with attributed sources, so when an individual gets a response to a prompt, they can then double check the source to ensure its accuracy. General AI models don’t tend to cite sources so accuracy can be questionable.

Some considerations when configuring AI tools include first and foremost, what is the problem you’re trying to solve. In the case of PwC’s tax assistant, its aim is to speed up the initial research process and cut the time it takes for a first draft, for example.

But the AI assistant will only improve productivity if it is based on in-depth, high-quality, accurate data with an ability to validate the source. The source of your data is the most critical part of configuring AI tools.

Mohbeen Qureshi, tech expert, data analyst, and VP of Growth at Oppizi, a New York startup, says: “It’s all about the foundation you’re building. Define the end goal, ensure the data reflects the tasks the AI will handle, and test along the way. If the data is skewed or incomplete, it can throw off predictions. You must also watch out for overfitting, which happens when a model performs well in training but struggles with new data.”

Continuous education for AI tools

AI assistants like these need to be dynamic though. Rules and regulations change so it is also critical to monitor and maintain these types of AI databases. Or else professionals run the risk of relying on out-of-date information.

Tinsley says: “You've got a number of different things to think through. Is the quality of the data complete and accurate, because if you've missed half of the legislation, you're going to get half the answer.”

The breadth of information is relevant too. Too much information could dilute the output. Maintenance of data sets is also critical, because if your database deals with tax legislation, that is constantly evolving.

The human comparison would be equivalent to a tax professional fulfilling their professional requirements to remain up to date on tax changes. The AI tool will have to undergo continuous development as humans do.

David Hunter, founder of Local Falcon, an SEO rank-tracking tool powered by advanced AI, says: “The amount of data you use also matters. While it’s true that more data often improves performance, it’s not just about volume, it’s about variety. If your data only represents one type of client or one kind of financial transaction, the model won’t generalise well to other situations. The trick is finding the right balance between quantity and diversity.”

Data questions

Once an organisation develops an AI tool though there are further questions to ask, such as who owns that data? Considerations over laws like GDPR and copyright restrictions are foundational too. Data confidentiality is a critical component in creating new AI assistants.

If an organisation creates a data repository by using client data and that database is accessible to the whole organisation, the AI tool may not know that it shouldn’t share or use this client data in other queries.

“The AI tool will potentially name clients, unless client names are removed. You’ve got to be really careful thinking through what information you're inputting and how you’re going to protect confidentiality,” Tinsley says.

In this case the best route is to anonymise or de-identify the dataset. Organisations are beginning to employ so-called data creation and curation teams, who will be responsible for the quality and accuracy of these databases.

Qureshi says: “Compliance comes down to knowing the regulations and sticking to them. Data anonymisation and consent are non-negotiable. For example, if you’re using client records, stripping personal identifiers and ensuring the data is legally obtained keeps you in the clear. It’s also important to document how data is handled, in case it needs to be reviewed.”

Bias is another factor to consider. Peter Wood, CTO, Spectrum Search, says: “Managing bias is essential. Historical data can introduce biases that skew results. Examine fairness in your dataset and employ fairness-aware approaches to minimise bias. A diverse review team can expose and address these biases, ensuring the model is fair and effective.

Regular auditing of AI systems to manage bias and ensuring they function correctly is a core component in keeping bias out of AI tools.

What is clear is that we are in the foothills of AI configuration. Dynamic iteration, learning from constant training of AI tools and regular updating and curation of high-quality data will ensure organisations remain on top of the risks we face from AI.