The PHE Covid-19 testing data debacle has created a great deal of comment and discussion. Leaving aside the rights and wrongs of the actual issue, some of the attitudes it has revealed have been illuminating.
In its analysis of the issue, the BBC reported the following comment from Prof Jon Crowcroft from the University of Cambridge:
"Excel was always meant for people mucking around with a bunch of data for their small company to see what it looked like,"
"And then when you need to do something more serious, you build something bespoke that works - there's dozens of other things you could do.”
"But you wouldn't use XLS. Nobody would start with that."
I certainly wouldn’t argue with Crowcroft’s conclusion that there were many better alternatives to Excel for such a vital project as the PHE track and trace system. It’s also possible that this particular comment was taken out of context. However, it does demonstrate the sort of contemptuous attitude to Excel and spreadsheets in general that has prevented organisations of all types and sizes from tackling the issues that spreadsheets create. It also seems to demonstrate a lack of understanding of the role of spreadsheets in the business world in particular. Far from just small companies using Excel to muck around with a bunch of data, the use of Excel underpins key systems in a great many multinational financial installations. In almost all businesses spreadsheets form a significant part of many business-critical systems. To assume that only small companies use Excel, misunderstands how businesses in the UK actually work. This impression is reinforced by the comment that, for something more serious, “Nobody would start with [XLS]”. Perhaps nobody should start with XLS but there is a very large number of people who definitely would, as the current crisis demonstrates.
It’s not so much the point that Crowcroft is trying to make that illustrates the issue, it’s the language he uses to express it: ‘mucking about with a bunch of data for their small company’. It would be hard to read this without concluding that the author holds spreadsheets in some degree of contempt.
This is far from an isolated example. I have spent many years trying to persuade all sorts of organisations to take spreadsheets seriously and have often come across the attitude that spreadsheets are just something that users play around with to the irritation of those responsible for the ‘real’ IT systems. To dismiss them in this way, rather than acknowledging what they can and can’t do, and what skills are needed to use them safely, is one of the main reasons why spreadsheets, misused by those lacking basic competence, can cause such damage.
One of the many dangers of the demonisation of spreadsheets whenever an error occurs that involves the use of a spreadsheet is the threat to the use of spreadsheets for the many purposes for which they are the best solution. It’s all very well to say that for anything serious you can ‘build something bespoke’ but I’d hate to even imagine the effect on business of waiting for the specification, development and testing of bespoke systems to replace all spreadsheet use.
Coincidentally, just before this particular error was discovered, an article I wrote for Tech News on spreadsheet errors was discussed extensively on LinkedIn:
The article considered whether 'spreadsheet' errors are really due to the use of spreadsheets or to human error that happens to involve spreadsheets. This is important because it determines attempts to find a solution. If there is something about spreadsheets that really means they cannot possibly be used safely for any important task, then we need to identify and publicise this to avoid the harm that these errors can cause. On the other hand, if a problem is caused by human error that would have occurred whatever system or application was in use, then getting rid of spreadsheets isn't going to help and we need to look more deeply to identify the real problem, rather than using spreadsheets as an easy scapegoat and convenient distraction.
The PHE data issue has highlighted the issue because it has resulted in very public revelations of a problem that potentially threatens all of us, rather than just resulting in financial loss for a particular organisation.
Given that we’re probably all on the same side and want to see the most appropriate applications used to solve business and government problems, it’s time to appreciate that the way to address the issue of bad spreadsheets is not to dismiss all spreadsheet use with a patronising sneer, but to understand why so many people, in so many organisations, turn to the use of spreadsheets. If we come to the conclusion that spreadsheets do have some role to play, then we need a serious discussion on how we can make sure that spreadsheets are used appropriately and that the people who use them are equipped with the necessary knowledge and skills to use them properly.
This is particularly true for organisations such as the main accountancy bodies that have an influence way beyond their actual membership through their members’ clients. Accordingly, it has been very encouraging to see ICAEW launch a series of projects to promote good spreadsheet practice and user competence, as well as issuing specific guidance on the ethics of ensuring adequate skills when offering spreadsheet services and advice.
At this time, it's hard not to wonder whether something as simple as general knowledge of the ICAEW Twenty Principles for Good Spreadsheet Practice might have created more awareness of potential problems and dangers and how to avoid them, saving not only reputations and careers, but also avoiding potential harm to public health.
Excel community
This article is brought to you by the Excel Community where you can find additional extended articles and webinar recordings on a variety of Excel related topics. In addition to live training events, Excel Community members have access to a full suite of online training modules from Excel with Business.
