Setting priorities in a corporate risk management program

Risk management is often based on qualitative judgements and qualitative ratings. For example, an evaluation of the likelihood of occurrence of a particular risk and its consequence may be made by reference to a rating scale. The Australian standard AS/NZS 4360 on risk management provides rating scales for qualitative evaluation of likelihood, consequence and resultant risk.

There are many factors and some difficulties involved in arriving at sound qualitative ratings of risk for example: the consistency of an individual's ratings; their degree of expertise in making a risk judgement; the complexity of a judgement (especially one involving multiple risk criteria); and the interpretation, scaling and use of ratings.

In general, people tend to overestimate the likelihood and severity of rare external events that they cannot control, like natural disasters. They tend to underestimate 'internal' events (like drinking and smoking) that they can control. They would tend to view the same risk differently if it was described say as a 20 per cent chance of loss compared with it being described as an 80 per cent chance of gain.

The purpose of this case study is to show how a specific technique for obtaining reliable qualitative ratings was applied to the assessment of risk in a commercial environment. The method was used to identify the various perceptions of risk in the organisation, to set risk management priorities in accordance with those perceptions and to direct resources to those risks perceived as the most significant to the organisation.

Setting and context

The second case is a large national clothing and sports good manufacturing company. It has manufacturing, warehousing, sales, distribution and administrative operations over a number of sites.

The company has comprehensive insurances but has yet to develop and implement a risk management strategy. The company retains high levels of self-insurance.

Its board is concerned to develop a risk management strategy that focuses on control of the company's major exposures to its business operations. Under its finance director, the company has prepared a budget for implementation of a risk management plan.

The finance director and the board wish to get a picture of what the key exposures to business operations are and the relative priorities. Using this picture, they intend to allocate the risk management budget and resources accordingly.

In their view, they wish to ensure that the company's resources in this regard are directed effectively to the risk areas of greatest importance and not wasted on areas of lesser significance to the company.

Risk identification

The organisation approached the development and implementation of risk management in five steps. These five steps are outlined below.

Approach

The company contracted an independent risk management firm to undertake an initial risk identification and risk evaluation.

For its initial identification of risks to business operations, the risk management firm firstly conducted a series of discussion (or 'brainstorming') workshops with different groups of company personnel.

Groups variously comprised the company's directors, its executives and general managers, line managers, operational supervisors, engineering and line representatives. In addition, the company's auditors and insurance broker were included. Workshops concentrated on identifying the exposures to business operations and the key criteria of significance to the business.

Secondly, the company's sites were inspected by a team made up of persons from the risk management firm, the company's insurance broker and auditor. The site manager was also involved in each inspection.

A qualitative risk profile of each site, including risks pertaining to its linkages to the overall business, was prepared from each inspection.

In summary, the risk identification workshops produced the findings tabulated below. The major threats to business operations were perceived as:

Any one of the threats listed above could produce a number of consequences and knock-on impacts to the business if it was to arise. In other words, there are a number of criteria against which risks to the business can be identified and assessed in importance. In this sense, the significant risk exposures to the business were perceived as those tabulated below.

The risks to business operations were thus seen as multi-dimensional, being a matrix or combination of threats that could feasibly arise and the resultant exposures of the business if they did. Table III below illustrates such a risk matrix.

In contrast to a risk rating based on an assessment of the likelihood and consequence of individual risks, this approach to risk assessment and rating of risks focuses on the threats and exposures (or vulnerabilities) to the business.

The question is how to rate the risks – that is, how to rate the relative importance of the threats given that there are a number of criteria against which they can be judged as a risk.

This rating task is made more complex if weightings are introduced for each of the criteria in respect to each individual threat. This is usually done, and it was in this case. For example, the criterion of business interruption got a high weighting for the threat of transport loss; the criterion of assets loss a very low weighting for this threat.

Note that the weightings shown in second table are the overall weightings for all threats – their derivation is discussed below.

Risk rating and relative risk priorities

Method used for risk rating

There are a number of ways to develop valid and reliable ratings from qualitative estimates of risk: for example, rankings and ordinal ratings, continuous linear scales, paired comparisons.

Paired comparison techniques have been used to arrive at numerical risk ratings (see Teniswood et al, 1993 for example). People make better relative judgements, such as with paired comparisons, than direct estimates. Paired comparisons enable a check on the consistency of judgements - this means for example , if an individual judges that risk A > risk B and risk B > risk C, then it should follow for consistency that risk A > risk C. In addition, paired comparisons can produce a numerical scale of results. The disadvantage of the technique is that it can be time consuming.

In this case, a method for rating risk using paired comparisons was adopted. First, paired comparisons of the six criteria shown in Table II above were made by all groups and their results consolidated to produce the relative weightings shown in the table. The paired comparisons were obtained using worksheets presenting 15 random pairs of each of the six criteria. The results of all comparisons were analysed using software for the purpose.

Secondly, worksheets giving random paired comparisons of the 11 threats shown in first table for each of the six risk criteria were presented to all individuals who had made up the risk assessment groups. A typical worksheet is shown in below.

All pairs were scored in the range 0-3 against the stated criterion. Each risk criterion was weighted as can be seen.

The relative expertise of each individual was also weighted in respect to the applicable risk criterion - for example, the marketing manager was given a greater weighting in regard to their assessment of risk of market opportunity loss than the assessment of the maintenance engineer say; and vice versa in respect of risk of business interruption.

Each individual indicated their judgement of the relative risk against each given pair on the worksheet, as the worksheet above illustrates. From the results of all worksheets, and taking all six risk criteria into account in the risk ratings, a risk rating between zero and one on a linear scale was produced for each of the 11 threats.

The relative risk ratings made by three of the groups of all 11 threats are shown on the linear scales below. The scales go linearly from 0 at bottom to 1 at top. It can be readily seen that there are different perceptions of the relative importance of the threats to the business. That is, different groups arrived at quite different ratings of the risks.

This is an interesting finding, though not really surprising. Different people had different perceptions of the risks to the company. Who is 'right'? And what are the 'real' risk ratings?

The overall risk ratings produced from consolidation of the results for all individuals are shown below. The bracketed numbers are scale values. This was used to direct the company's resources to the risks seen (by all) as being of greatest significance. The resource allocation was based on the relative rating (scale value) of the risks.

Guidelines

This case study suggests the following guidelines for qualitative assessments of risk and risk management:

• Different people have different perceptions and understandings of what risk is
• Different circumstances and different questioning produce different assessments of risk
• Though ranks and direct ratings are easy to use, people are not particularly good at making direct (or absolute) estimates of risk
• Numerical scores to indicate rank and direct ratings provide ordinal data only and should not be treated and used in calculations as real numbers without care
• A paired comparisons method can be used for risk rating and its scale values show not only the relative rating of each risk, but by how much. Results can also be checked for consistency and reliability. The disadvantage of the technique is that it is time-consuming and generally requires software to help
• Thus, the risk manager requires an understanding of the strengths and limitations of qualitative data, qualitative techniques and the interpretation of qualitative results.

References

Teniswood CF, Sharp T and Clark DGN Case studies in probabilistic risk assessment in Melchers RE and Stewart MG (Eds.) Probabilistic Risk and Hazard Assessment, Balkema Publ., 1993, pp. 111-119.

Back to case studies

This page is available online at:
http://www.cpaaustralia.com.au/cps/rde/xchg/cpa/hs.xsl/2742_3545_ENA_HTML.htm