When the Financial Reporting Council (FRC) published a thematic review of audit sampling (in November 2023), it made some key recommendations for all audit firms, offered insights into sampling methods and methodologies, and shared observations on common practices, good practices and concerns. This article explores what the findings mean for smaller firms and highlights some opportunities to enhance audit sampling.

Although this FRC report on audit sampling and other audit thematic reviews are based on what the FRC finds in some of the largest UK audit firms and networks, these narrow-focus thematic publications have wider value across the audit ecosystem. Some of the findings from this thematic that caught my eye relate to aspects of audit sampling where file reviewers from Insight Training also find shortcomings and room for improvement. 

Take judgements in respect of sampling, for example. These are often poorly documented, according to the FRC, and I’m inclined to agree. It can be tempting to think that auditor judgements apply only to accounting treatments and estimates, but professional judgement also applies to things such as materiality and sampling. If you’ve applied judgement to arrive at a sample size that is different to what your sampling tool suggests, it’s important to make the thought process behind that judgement clear on file.

The FRC also found that auditors are not always clear about the basis for reliance on alternative audit procedures to limit sample sizes for tests of detail. This is a recurring theme for many firms, especially in respect of substantive analytical review (SAR), which must be ‘predictive’ for reliance to be placed on it. Interestingly, among the firms the FRC reviewed for this thematic, auditors often directly link SAR reliance to the allowed tolerable difference between the actual amount and auditor’s expectation.

The review highlights that care should be taken when picking ‘key items’ in a stratified sample. Reasons for selecting key items need to be clearly documented and explained, but the FRC finds that they are rarely recorded. When they are, the FRC observes that this usually states simply “selecting everything over 50% of performance materiality”, with little or no justification as to why 50% is a meaningful percentage. If auditors take the time to carefully identify risky items, the remaining population will be lower risk and result in a smaller residual sample size – another example of a potentially key judgement that needs to be explained on file. 

Testing information produced by the entity (IPE) and attribute testing also offer room for improvement. According to the FRC, confusion over the function of testing IPE and objectives of attribute testing are common among engagement teams. The thematic report dedicates a section to associated sampling methods, which includes clarification on concepts and other information that some smaller firms may find helpful. In my experience, not challenging hard enough on the integrity of IPE is an issue that crops up on many cold file reviews.

File reviewers often challenge firms on how samples are picked, as well as on whether sample sizes are big enough. To be able to draw inferences about an entire population, each item in the population needs an equal chance of being picked. The FRC thematic highlights the benefits of random sampling over haphazard sampling – but a perhaps more fundamental question concerns whether audit files make clear how each item has had an equal chance of being selected.

There is a section in the thematic dealing with sample selection for tests of control, which some of the top tier firms (reviewed for this thematic) have placed “enhanced reliance on”. At Insight Training, our experience of smaller firms is that limited reliance is placed on internal controls – not least because standard proprietary systems seem to give very limited ‘credit’ for controls tested when determining sample sizes for tests of detail.

The FRC review also highlights the importance of firms being able to understand the statistical concepts behind their sample size methodologies, particularly given requirements in the new and revised International Standards on Quality Management. Audit firms that rely on working papers from methodology providers may want to revisit this.

Even the smallest audit firm may also want to reflect on the ‘key recommendations’ in the FRC thematic, which reflect the significance and impact of the auditor’s professional judgement throughout the sampling process. The review found all audit firms should:

• ensure that they provide engagement teams with sufficient guidance and training to support their use of professional judgement in audit sampling; and
• update their methodologies and guidance to drive better documentation of key professional judgements in this area.

What is not in the audit thematic may also interest many smaller firms. The ‘elephant in the room’ is the use of sample size caps, especially in areas of significant audit risk. This does not get a mention at all, despite previously being raised as a significant thematic weakness by the FRC in its Developments in Audit 2021 report, which led at least one of the major methodology providers to completely dispense with sample size caps. This has had a huge bearing on sample sizes – especially for transaction tests – and made firms think very hard about their risk assessments and possible reliance on non-sampling procedures.

This has been reflected in faculty resources, such as a 2023 article Sample sizes and caps: how much is enough? and a related Q&A. 

Making sense of sampling and substantive analytical review is the subject of a 2024 faculty webinar on approaches to SAR.

ICAEW’s professional judgement hub brings together a range of resources, insights and expert insights to help you develop your skills in approaching difficult judgements.