Businesses and risk managers are regularly faced with making decisions under uncertainty. For example, decisions such as:

• Is $50 million sufficient insurance cover for a particular project or a particular contract?
• What is the chance that a system or a procedure might fail or a product might be defective? Or a decision or a piece of advice might turn out to be wrong? And what do we do about it?
• What is the feature or aspect that is most likely to fail? Thus where do we focus our effort to reduce the risk of failure?
• Should we spend money on implementing a particular risk improvement, like extending a sprinkler system during building refurbishments or purchasing extra insurance cover offered by the broker? What's the cost-benefit?

The purpose of this case study is to examine how a particular company applied the concepts of risk and risk analysis to making improvements to product quality and its production processes. Specifically, this case looks at how the company identified the risks of inadvertently manufacturing a faulty product, established where best to focus its risk improvement efforts and then justified expenditure on improvements to reduce the risk.

Data presented in this case study is for illustrative purposes and is fictitious.

Setting

The company in this case is a manufacturer of a range of food and beverage products. It supplies these products to retail, wholesale, export and food service markets. The company's products are distributed across a variety of brands. The company also imports a number of products and distributes these under licensed brand names, primarily through wholesalers.

The company has had to recently recall a large batch of one of its major retail product brands. The product was defective in flavour and texture, and thus not of merchantable quality, though there was no risk to consumer health. This problem arose as a result of a shortcoming in the production process. The costs to the company associated with the recall and loss of product were substantial, about $0.3 million.

The costs associated with loss of brand image, market share and opportunity have not yet been quantified but are expected to be significant. In response to this recall, the company has instituted a detailed appraisal of the production risk to see where improvements to risk controls can be made.

Background

The company has well-established quality control and quality assurance procedures. The procedures include a hazard analysis and critical control points (HACCP) process. The company has extended these procedures across its suppliers, distributors, warehousing, maintenance, and transport.

Moreover, the company reviews its risks and risk management program annually. As part of this, it identifies those products and product changes that pose potential liability risks in the light of legislative developments, community expectations, changes in markets and the company's review of loss performance and any customer complaints.

The Australian standard on risk management AS/NZS 4360 recommends that the risk context be defined. Product liability risk in this case may be seen in the following context. The Trade Practices Act has established a regime of strict product liability – that is, a product is deemed to have a defect if it is not such as persons are generally entitled to expect. The defect may be in the design or manufacture of the product or in the instructions associated with the product. In addition to any breach of the Act, the consequences of a product defect for the company are damage to its reputation and market standing, loss of customers, cost of loss and liabilities, and cost of product recall.

The company believed that a risk management approach would give a fresh view and added benefits to its product quality/safety procedures and its present qualitative HACCP program. In particular, a risk approach might quantify its product-related risks and show where risk reduction efforts would be best applied on a cost-benefit basis. This was especially to reduce the risk of rare events of high consequence, such as a product safety or health problems.

Method

The risk approach comprised the four steps as follows:

• Examine the production process and identify the key risk elements that could possibly give rise to process failure and a downstream product failure, or defect
• From this, highlight where risk reduction measures might be best applied
• Compare the change in level of risk before and after risk control measures
• Undertake a risk-cost-benefit assessment and justification of the risk control measures.

Product risk identification

The first step of identifying potential product risk scenarios was undertaken by a group of company personnel consisting of process operatives, supervisors, managers and technical staff who were expert in the production process.

In essence, this step involved a 'brainstorming' session, using the knowledge of the group members plus the various process and product analyses provided by HACCP and other existing QA/QC information.

This risk identification was structured using a 'fault tree' approach. A fault tree enables an adverse event (like occurrence of a product defect) to be analysed by defining the different fault scenarios leading up to that event and quantifying their probabilities of occurrence.

Risk evaluation and quantification

Once the fault tree for the specific product defect was completed, the probability of occurrence of each fault or failure within the tree was determined.

These probabilities were obtained from estimates made by the group of company experts, from process failure data where it was available from QA/QC and other production records, and also from the logical calculations afforded by the tree. An example of a question used to help with this probability estimation is shown below.

The typical measure of failure probability was faults per year (FPY). For example, a failure level of 0.2 FPY indicates this failure will occur on average once every five years.

A specific risk modeling technique (Teniswood CF et al, Case Studies in Probabilistic Risk Assessment, Balkema Publishers 1993 pp. 111-118) was used to help quantify each individual's judgements and consolidate the group's results.

Depending on the availability and quality of records for individual faults and failure rates, production fault and failure data were augmented with the expert judgement data using a simple Bayesian updating technique. That is, the production failure-rate records were combined with the expert judgements to form a better estimate of failure probability.

For example, if the expert judgements indicated an average failure rate of 0.4 in 7.5 years (or about one failure in 19 years) and the company's records showed one failure in 12 years, the updating method arrived at an average failure rate estimate of 0.072 or one in 14 years.

The lower level probabilities were combined using logical calculations (set or boolean algebra) in accordance with established fault tree methods to produce the probability pertaining to their higher outcome. For illustration here, in simplified terms these calculations basically consist of adding probabilities where a logical 'or' applies and multiplying probabilities where a logical 'and' applies.

For example, the fault at node 18 (material wrongly graded as okay, with 0.16 FPY) or the fault at node 19 (rejected material is inadvertently used, with 0.04 FPY) could occur. Either would lead to the fault at node 17 (undetected excessive bacteria) occurring with a probability of 0.16 + 0.04 = 0.2 FPY.

Though the above quantifications may sound daunting, they were readily and quickly conducted. The whole risk analysis exercise took less than 2 weeks to do.

Results

The estimated probability of sub-standard products getting onto a retailer's shelf was 0.238 FPY or about once in four years. The fault tree showed a number of scenarios contribute to making a defective batch in plant.

Risk management guidelines for this case study

This case study suggests the following guidelines:

• Key characteristics of risk are likelihood and consequence. The likelihood of a risk arising, like what could go wrong with a product, a strategy or a service, can be systematically evaluated using techniques like fault trees
• Such systematic risk analysis can show where risk control efforts might be best directed to reduce the risk. Reducing likelihood reduces risk
• The decision of whether to adopt risk control measures or not in the face of a situation with various options might be based on one or more of the following criteria:
• Taking the lowest expected cost of the options available
• Taking the option which minimizes the worst case impact
• Taking the option which represents the most likely outcome
• Assessing whether the impact or chance of each option is acceptable or not.

Back to case studies