We’re all used to getting feedback surveys asking for our opinions about products, services and experiences. Sometimes we complete them, sometimes we don’t; for many different reasons. One of the main motives we have for completing them is because we want our views to be listened to and acted upon.
When ICAEW’s quality assurance reviewers carry out their reviews of firms and insolvency practitioners (IPs), the aim is to ensure that the expected standards and regulatory requirements are being met. But these reviews are also about supporting firms in meeting those requirements.
“We want to make sure we’re carrying out our regulatory role effectively, so we also need to know how the process is working for the firms and IPs on the other side of it,” explains Bob Pinder, QAD Director. This is why the survey asks questions about firms’ overall experience, the quality of the interaction, and the support provided by ICAEW.
Temperature check
“We’re genuinely interested in what the firms think of the review process,” says Michelle Giddings, Head of AML and Operations, QAD, at ICAEW. “It provides a useful temperature check as to whether we’ve got the balance right of all the different models and delivery methods we use.”
To this end, one of the questions is: ‘Did you feel the review was the most appropriate for your firm given the nature and size of your practice?’
Monitoring visits are carried out by QAD, part of the Professional Standards Department, which delivers ICAEW’s regulatory responsibilities.
“As an improvement regulator, QAD visits form quite a big part of the training and guidance we deliver, and we need to know whether that’s effective. The questionnaire is a really good measure for us,” says Michelle.
“Our reviewers visit lots of different firms, and they can play back that experience during reviews. We find firms really appreciate talking to someone who's been into other firms, seeing how they're grappling with a particular problem,” adds Bob.
One respondent to our most recent set of surveys put it this way:
“I think that the review went well because of the approach taken by our reviewer which came over as 'we want to help you to be better' rather than that we are simply checking whether you do or do not meet 'the rules’.”
“We also link the findings to ICAEW’s ways of working, its key values and what we want to be as a regulator,” explains Bob. “It's not just a list of questions like a standard feedback survey; instead, it puts it into the context of the kind of organisation ICAEW is aiming to be.”
Some of the questions, for example, are designed to draw out information about whether reviewers showed initiative, acted with integrity, and provided insight into the firm's business.
The feedback received also helps ensure ICAEW’s activities as a regulator are proportionate. “We've got to be proportionate to the risk, “says Michelle. “And survey feedback is a good barometer to make sure firms feel our input is fit for purpose. What we find is that usually they do think that and, if anything, they probably want more contact.”
“But if the feedback from a large number of firms was that ‘this is overkill for my practice’, we would reflect on that,” she emphasises.
More respondents
Every survey response is collated and analysed by an independent research agency, which produces a quarterly overview report.
Figures for the second quarter of 2023 have recently been published on our website and show very positive feedback, with 95% of responding firms satisfied with the management of the process and the interaction with QAD staff. More specifically:
- 82% were ‘very satisfied’ with the management of the process (Q4 2022 77%); and
- 85% were ‘very satisfied’ with the quality of interaction with QAD staff (Q4 2022 84%).
Reviewer performance ratings also remain high, ranging between 84% and 91% across the five criteria of having a good business understanding, and being constructive, technically competent, patient and courteous.
For example, one respondent said:
“I cannot thank the reviewer enough for the way she approached everything. She was thoroughly professional and efficient, had clearly done her homework before and during the review, but was empathetic.”
This and previous results indicate that the process is working well. But to ensure the data continues to give the best possible picture, ICAEW needs more firms to get involved, complete the survey and provide their perspective.
Make it count
The higher the response rate, the more reliable, robust and valuable the data becomes. In the most recent quarterly report, the response rate was 26%, which is back up to pre-pandemic levels.
“This is good news, but we’d like to see it higher to get the best possible indication of the true nature of the feeling out there,” says Bob
The survey is sent to you at the end of the visit process and is included as a link in your closure email that explains that QAD has concluded your visit.
“We really urge you to click on the link provided and complete the survey,” says Michelle. “One of the issues we’re noticing is that people might click to start the survey, but then fail to finish it.”
“We've done a lot of work improving the questionnaire’s presentation to condense it down,” she adds. “It only takes around five minutes and there are about 30 questions, which require firms to tick a range from very satisfied to unsatisfied.”
The questions are simple to answer, and there are also freeform sections for firms to give additional feedback on anything they’d like to raise.
“You don't have to write much freeform if you don't want to, but we really value those verbatims in terms of letting reviewers know what is working and what isn’t,” says Michelle.
“And if there's a theme coming through the narratives, we play that back at team meetings and people will talk about it, so we’re confident that if there are common issues coming up, we'll address them.”
The questionnaires are anonymous. But at the end, firms can also put in their contact details. “If there's something about the visit you feel so strongly about that you want to have an independent conversation about the process, then that's a good way for you to get that interaction,” says Michelle.
It's not too late
If your firm has experienced a recent monitoring review, and you haven’t completed the survey, it’s still possible to access it and complete it. Your input and feedback are always welcome.
Sharing the insights
QAD tracks trends and actively uses all the survey data to support the continuous improvement of its monitoring processes and procedures.
“We always take the answers to the questions very seriously,” says Michelle. “They don't just go into a black hole. We circulate them to the entire review team, and we'll discuss them in team meetings. And if there are any themes coming out in the narrative, senior managers will talk it through with reviewers.”
The reports on the findings go directly to the ICAEW Regulatory Board (IRB) and to the ICAEW Council, and the headline results are included in the Regulatory and Conduct Annual Report.
It’s also important that firms get to see the results of the findings themselves. So, the quarterly figures are published with infographics online and highlighted in ICAEW’s email bulletins, including Regulatory and Conduct News.
“We want our regulatory approach to be proportionate and fit for purpose to achieve our objectives in the most meaningful way for everybody,” emphasises Michelle.
“Filling in the feedback survey is one way that firms and IPs can really make themselves heard,” she adds. “It’s a valuable opportunity to have direct input into what good regulation looks like and helps shape a regulatory environment that fosters improvement and benefits the profession, clients and the wider public interest.”
Resources
Keep updated
Be the first to know when articles like this are released by following us on LinkedIn and subscribing to our monthly newsletter, Regulatory & Conduct News.
