The return on investment (ROI) made in people-related programmes, such as HR projects and training programmes, is notoriously difficult to assess. Whereas investment in IT or quality can normally be targeted as cost savings or measurable customer benefits, the objectives of many HR programmes are more diverse. Some indeed are undertaken more as an 'act of faith', an intuitive belief that the programme will benefit the business either short or long term but with the knowledge that it cannot be directly proved.

The heart of these acts of faith is a belief in a chain of cause and effect that will eventually lead to a bottom line benefit. If there is only one link in the chain, the task is relatively easy. However in the HR area this is rarely the case. The immediate objectives are often not financial. Each link in a chain brings contaminating factors, ie other influences which dilute the effect of the original intervention. To evaluate any return that is at least credible financially, we not only have to establish realistic links but also to be skilled at assessing the financial impact or otherwise of non-financial benefits along the way.

The most prominent writer in this area is an American called Jack J Phillips, who has numerous books in this niche field. He identifies a number of factors that make most HR and training people shy away from doing full ROI assessments. One of these is a lack of both interest and skill in numeracy and complex assessments of cost and benefit. A second is the availability of time. It is often a complex data collection exercise that is required to evaluate returns over a period, making it a project (and therefore cost) in itself. If the discipline needed was not planned for in advance, it may prove to be very frustrating as well. It is much more akin to human nature to just move on to the next initiative. Besides the results may not be very encouraging, and therefore there is an element of fear perhaps of being hoist with one's own petard.

Apart from the difficulties, perhaps the most common reason for the lack of ROI assessments is the absence of pressure from senior management. Most HR and training professionals will express interest and concern that they should be better at it and do more of it, but few organisations demand it of them in any rigorous way. There are exceptions, particularly some very measurement-orientated American companies, but it seems that most organisations, private and public, are more concerned to agree a budget for support functions and let them spend it in the way they feel is best. In practice, as in IT and marketing, a lot of money can be spent for little gain.

This might seem like a strange question, but value can be both financial or non-financial. As an individual I spend money on, for example, a holiday to receive a number of different non-financial returns - time with my family, experiences, relaxation and so on, and at the end of the holiday I intuitively make an assessment as to whether what I got from it was worth what I spent. Even in business we spend money for non-financial gain - prestige, reputation, building political positions for example. Sure, we believe they are in the long term interest of the bottom line but we would not attempt to prove it pound for pound. It is a matter of judgement whether the gain - expected or realised - is worth the investment, and there will probably be different opinions about that.

In the fields of training, organisational change and HR initiatives we are more likely to be aiming for non-financial benefits, in the first place at least, than for a targeted bottom line gain. Where projects have direct goals such as reducing labour turnover, or absenteeism, or increasing productivity or safety - then the benefits can be translated fairly easily into financial gain. We need to be careful not to claim savings which are not actual cash - the classic error is to accumulate time savings and assume it is real money. (In theory it may release time to be spent on other, more value adding, things but this would be very difficult to estimate). If we are investing in culture change, diversity awareness, strategic understanding, team building or even leadership development financial gain will be much more difficult to assess. One government department spent a million pounds with a prestigious business school and the stated aim was to enable the top 100 managers to manage major change effectively. How could this ever be evaluated in ROI terms? The managers certainly enjoyed it. But...

Donald Kirkpatrick did his PhD thesis in 1959 and little did he know that he and his methodology would be a household name even 50 years later as the classic methodology for 'levels of evaluation' of training programmes. He identified four levels - the first was the experience itself, followed by evidence of real learning. The third was application of the learning to the workplace and finally 'level 4' the effect on the bottom line. Jack Phillips added ROI as 'level 5'. It has always seemed to me however that there is a level we might call 'level 0' which is fundamental, and that is the setting of the objectives of the programme.

The specification of what we are trying to achieve - clearly and measurably - will in itself determine the nature of the benefits we will assess. If we are 'investing' in team-building, for example, it may be that our objective is that the team will make better decisions as a result. But we will never know that for sure. We will probably have an objective that team members felt good about the event and it 'increased morale'. That might be much easier to assess, as well as be the real goal anyway. Objectives therefore may be at any of the 'levels' - whichever one it is dictates the level of cost/benefit balancing. Say, we spent £1,000 to increase the Excel skills of 10 people to 'Microsoft master' level. At the end of the day we would ask ourselves - was £100 a person worth it to get that level of skill?

It can be argued that everything can be measured in some way. Morale, understanding, competence, commitment, culture change, reputation - however intangible. We need some carefully designed rulers but they can be created. Culture, for example, can be measured on a template of characteristics that describe what we are trying to be like - and from time to time we then collect perceptions of what people are experiencing. If this holds true, we can express objectives of all 'soft' initiatives in quantifiable ways, often as a percentage change in a chosen measure. If we have clear measurable objectives, it is but a short step to assessing whether they were achieved.

The problem is that, in many HR-related initiatives, we do not have these. They are often vague, generalised, aspirational hopes. This is nearly always true on one of today's major expenditure areas, namely 'leadership development'. What exactly are the visible results we seek from our investment? The perceptions of leaders by their 'followers'? Improved morale and productivity? Better decisions? How do we measure these and what percentage change from the current status do we seek?

When such programmes are evaluated one usually finds that the individual participant feels very positive about the experience. They have gained personal insight and feedback, and many will talk of their intentions to lead/manage in a different way. However the realisation of that in any tangible measure is much more difficult. The fact is that they go back into the culture and organisational system they left and the intention to change is often subject to slow erosion.

Some programmes are justified by 'group projects' - by giving participants a task to solve or an area to innovate in. There is plenty of evidence that groups do come up with ideas that benefit the bottom line during such development programmes. The programme has provided the platform, although one could argue that the provision of time and incentive could have been done without it.

The longer and more complex the cause and effect chain, the more we will have 'contamination factors' - or 'additional influences' on the end benefit. Philips goes to some lengths in discussing how these can be isolated. The clearest option is a control group. Thus we could compare two similar groups of leaders, one group going through a development programme and the other not. Ronald Burt and Don Ronchi (2007) reported on such a study carried out in the Chicago Graduate School of Business in relation to their Business Leadership Programme. They concluded that "programme graduates are 36% to 42% more likely to receive top performance evaluations, 43% to 72% more likely to be promoted, and 42% to 74% more likely to be retained by the company." They also said however that statistically the level of significance of these results was low.

Other methods for isolating causes are to use trends and forecasts and to see how they vary from what would have been expected without an intervention; manager and participant perceptions of the effects, or using an external expert study. All of these approaches take time and money and are not to be undertaken lightly.

Investing in balance sheet assets follows a well-defined discipline. The costs of a project are assessed. There are always traps in under-costing, especially not allowing for hidden costs. Then the financial returns are evaluated over the life of the project. They are always best estimates and no manager usually gets away with a proposal where his or her assumptions are not challenged. A hurdle of return must be reached for the capital to be authorised. The real work is in the thorough analysis of the costs and potential benefits that will stand up to scrutiny.

Such disciplines are rarely demanded of 'soft' or 'intangible' projects, and yet the rationale for doing so is clear. This arises partly from the questionably rigid distinction between 'capital' and 'revenue' expenditures in today's knowledge-based world, and of course from the often non-financially quantifiable benefits. True, a 'business case' may need to be made for an initiative but it is the rigour of analysis that is often missing. If the effort was made to justify and scrutinise projects before they started we would rarely want to go to the difficult job of trying to assess the return on investment afterwards.

Nevertheless, just as a capital expenditure of more than a certain limit will require a full appraisal, so we can define some criteria which would guide us on whether a full post-project evaluation is likely to be worthwhile. Clearly the amount of expenditure would be one factor - the threshold would vary with the size of the organisation, and might be set at a certain percentage of the department's budget. A second criterion might be that the outcome of the project is strategically important and we really do need to know if it was successful. A third, particularly in the public sector, might be that there is a need to prove to other parties that money was well spent.

Some experts in this field insist that the only valid assessment of any project, HR or not, is in financial terms. Not only does this not apply to the not for profit sector, but it is unrealistic. The pursuit of such an approach leads to assumptions that may stretch credibility. This does not mean we should abandon the inescapable discipline of knowing whether or not we receive value for money. It requires first the development of expertise in measuring intangible benefits, clear measurable aims and objectives for programmes, and sound judgement in balancing costs and benefits. And, as an HR professional, I would make a plea to finance people to be more demanding on the expected returns - and helpful in providing expertise - for those projects which are aimed at 'intangible' benefits.

• Keith Bedingham, 'The measurement of organisational culture', Journal of Human Resource Management, January 1999
• RS Burt and D Ronchi, 'Teaching executives to see social capital: results from a field experiment,' Social Science Research, 2007
• Tom Granoff, 'A white paper' on the ProCourse method of measuring success, ProCourse Scientific Advisory Board, 2001
• Paul Kearns, Maximising the RoI from training - measure the value added by employee development, FT-Prentice Hall, 2000
• Paul Kearns, Evaluating the ROI from learning - how to develop value-based Training, CIPD, 2005
• DL Kirkpatrick, 'Techniques for evaluating training programmes', Evaluating Training Programmes, Alexandria, VA 1975, American Society for Training and Development, pp 1-17.
• DL Kirkpatrick (compiler), Another Look at Evaluating Training Programmes, ASTD, 1998
• Andrew Mayo 'One Stop Guide to HR Return on Investment', Personnel Today, 2004
• JJ Phillips, RD Stone and PP Phillips, The Human Resources Scorecard - Measuring the Return on investment, Butterworth-Heinemann, 2001
• JJ Phillips, Return on Investment in Training and Performance Improvement Programmes, Houston, TX: Gulf Publishing Company, 1997

Download PDF version:

• Finance & Management Magazine, Issue 159, October 2008

The ICAEW Library & Information Service provides access to leading business, finance and management journals, as well as eBooks. These resources are available to logged-in ICAEW members, ACA students and other entitled users subject to suppliers' terms of use.

Further reading on training and performance management is available through the resources below.