The effectiveness of risk assessments in informing decision makers

Purpose: This research examined the effectiveness of risk assessments in an organisational setting and found that risk assessors may not effectively assess risks and so may not provide the best available information or effective advice to decision makers. However, guidance on the conduct of risk assessments is scattered across many organisational, economic and technological domains with little consistency in definitions and processes, perhaps contributing to ineffective risk assessments. This is an area of academic and practical relevance that can be investigated rigorously and has the potential to improve decision making. Design/methodology: The main research question was: RQ. Can a generic goal tree, populated with critical success factors and necessary conditions derived from the literature and case studies, be developed that would help decision makers and risk assessors to evaluate the effectiveness of risk assessments before they are used to aid a decision? The research described here used a pragmatist approach to managerial practice and associated practitioner problems related to (1) how effectively risk assessments informed decision makers before decisions were made; (2) how formal risk assessments were carried out; and (3) how risk assessments might be improved. Following a comprehensive literature review, an online survey was used to determine what methods are employed currently, while case studies investigated the conduct of risk assessments in context. The literature review indicated that risk has many connotations and diverse definitions and conceptualisations, leading to the conclusion that a given risk assessment should state the definition of risk used. The ISO31000 definitions of risk ("effect of uncertainty on objectives") and risk assessments were found to be most useful for this research. The online survey of risk and safety practitioners found little use of structured techniques or consideration of human factors, a finding also reflected in the case studies. Potential critical success factors and necessary conditions for an effective risk assessment were identified from the literature review and linked using the Theory of Constraints logic processes to form a tentative goal tree (GT) including the entities and relationships underpinning effective risk assessment. This also provided a means of structuring and reporting the online survey data, and critiquing a pilot study and five further case studies, and selected professional practice developments. The case studies explored in detail how risk assessments were carried out in support of a management decision. Each case was investigated using document and literature reviews, and structured, one-on-one interviews, including review of the tentative GT by interviewees. Qualitative data from the case studies was analysed using NVivo and quantitative data from the online survey was analysed using SPSS. Findings: This research demonstrated the ontological and epistemological basis of the GT, justifying it as effective research tool and enquiry system and the goal tree diagram as an effective problem representation system. Together, the goal tree process and diagram provide guidance to practice, and in this research, act as a qualitative tool for predicting the effectiveness of risk assessments. Research limitations: This work was limited to an online survey, a pilot study and five case studies to explore risk assessments in an organisational setting. However, the findings corroborate anecdotal evidence that risk assessments are not effective, and the need for mechanisms for improvement. Practical implications: This research has identified a need for training in risk assessments in New Zealand. Mechanisms for improving risk assessments are identified, with one (a ‘risk canvas’) being developed for and tested during a series of training workshops and found to improve performance and confidence (but not consistency or reproducibility of risk assessment results). Other proposed mechanisms include use of the McKinsey 7-S model in gap analysis; open access research evidence; and further development of the goal tree. Originality: The research used a single goal tree and magnitudinal ratings of necessary conditions to judge the effectiveness of risk assessments in five organisations. The risk canvas to aid basic risk assessments was shown to provide improved risk assessment process