Наши популярные онлайн курсы
IIA recently published a Practice Guide on Assessing the Risk Management Process. It sounded interesting so I set aside some time to go through the document.
Overall verdict
The maturity model proposed by IIA has exactly the same fundamental flaws that all of the other 100+ consulting risk maturity models out there have. Plus, while correct messages are sometimes used throughout the text, more a coincidence it seems, the authors are still clueless to the differences in risk management 1 and risk management 2 and fail to provide useful advice on really assessing the risk management effectiveness. The practice guide is only usefull for assessing risk management 1, which has very limited value to the organization. Here is an alternative approach to assessing risk management maturity: https://www.youtube.com/watch?v=4HqVIpFw230 (7 mins)
Context
In order to understand the most fundamental flaw in the IIA take on assessing the risk management effectiveness you need to keep in mind the following:
- AS4360, COSO ERM and ISO31000 are not the birthplace of risk management theory, the idea to measure uncertainty and it’s effect on decisions and objetcives started in XV century (probability theory), then became prominent in 1970s in the field of decision science and neuroeconomics. Internation standards are basically mediocre summaries of the probability theory, decision science and behavioral economics in layman terms. To understand the concept of risk management effectiveness we need to look beyond the standards to the origins.
- there is a huge difference between risk management 1 (done as an element of corporate governance for the Board / Audit Committee, made popular in 2000s) and risk management 2 (where risk management is used a management decision making tool, with or without help from risk managers, existed since 1930s). More RM1 and RM2 in this article.
- regardless of what risk managers are doing, the management continues to use and always used proper risk management when making decisions (finance teams always use scenarios, investment teams always use sensitivity analysis, ingeneers always use simulations, procurement and legal use simple scoring, strategy uses checklists, etc, etc) – this is the most critical point in understanding risk management effectiveness.
- in fact most risk management in the organization is never called risk management. That is why risk management effectiveness has almost nothing to do with whatever is in the risk management policy or in the risk appetite statement (both documents are RM1).
Detailed comments
The document really gets off to a bad start:
Internal auditors must evaluate the effectiveness and contribute to the improvement of risk management process (Standard 2120 – Risk Management).
Interestingly, Standard 2120 actually says risk management processes, which is correct, but the authors, probably not realising the significance, called it risk management process, singular. This continues throughout the document and shows that authors probably don’t even realise that in the context of RM2 (risk management as a decision making tool), there are multiple risk management processes, often totally different. Nor should there ever be a single consolidated risk management process.
Then there is a moment of brilliance, but lets see if IIA can move beyond the slogans.
Risk culture: Integration of risk into all decision-making, compensation, reward structures, and goal-setting.
The next sentance is nothing short of strange, participation by eveyone in the risk management process. This has nothing to do with decision making, nor it is how decision making works in real life.
Risk governance: Participation in the risk management process throughout the entire organization by personnel that are knowledgeable, skilled, and competent in risk management.
Risk management process: Aggregated risk identification, prioritization assessment, treatment, monitoring, and reporting throughout the organization.
Then it gets even stranger, realiable data hard to obtain? Challenging? Nothing could further from the truth.
Measuring the benefits of a mature risk management process may be challenging because reliable data may be hard to obtain, if it is available at all.
The culmination of the practice guide is the high level maturity model below:
How mature should an organization be? Consider a scale of 1 to 5, with 5 being the most mature. It is not necessarily optimal or practical for all organizations to be operating at the highest level of maturity. Achieving a solid 2 or 3 may be acceptable. Each organization should determine which level of maturity is optimal for its circumstances.
The overall maturity model has the usual flaws of common maturity models:
- 1-3 levels have very little to do with effective risk management. Levels 4 and 5 attempt to summarise what an effective risk management may look like when it is integrated into business processes and decision making.
- then IIA makes the same mistake everyone makes by saying “achieving a solid 2 or 3 may be acceptable”. Obviously achieving level 2 or 3 is the same as not having any risk management in place. Unless risk management is valuable decison making tool which contributes to the key decisions made by the business, there is no point.
Risk Appetite
For many organizations, risk appetite is difficult to articulate for practical use in discussions. A common form of risk appetite is a statement of “loss tolerance” that may be approved by senior management and/or the board, with a caveat that the loss limit may be exceeded with approval by those with appropriate levels of authority.
Once again, IIA completely misses the plot on the whole idea of risk appetite. I have written so much about it already here: https://riskacademy.blog/?s=risk+appetite
Overall, this little section is pretty illustrative of the misconception in the whole document. IIA argues that if there is no risk appetite statement or something that resembles it, the maturity level is low. Which of course is nonsense, because business had documented its appetites to various decisions in existing policies decades before the very concept of risk appetite was created (segregation of duties, financing and deal approval limits, procurement criteria, investment criteria, zero tolerance to fraud or safety risks, etc).
Roles and Responsibilities
Seems fine, didn’t find anything too questionable in this section.
Culture
Seems fine, didn’t find anything too questionable in this section.
Governance
In general, the risk management process is developed from the top down, with senior management and the board calling for risk assessments and reporting first, typically leading business management to adopt the same practices later as it must provide the risk information for senior management to use.
Risk management is a decision making tool, it could be used for strategic decisions or operational decisions, it doesn’t matter, nor is the senior management calling a prerequisite.
Process
This is a very good example and I wish this is where the IIA would spend most of their time on:
The degree to which risk management activities are integrated with other business processes is a useful gauge of the organization’s maturity level.
But alas, it is immediately followed by this poor example:
I looked only the level 5 maturity because this is suposedly the desired state and it’s perfect for checking whether the authors have the right ideas in mind. Unfortunately, this vision for the mature risk manageent has nothing to do with decision science or probability theory. Use of risk information section is the only kind of close, but still missing the whole idea of risk management as a decision making tool.
Role of Internal Audit in Risk Management
This is a good example:
…the internal audit activity may gather the information during multiple engagements and may consider the results of these engagements cumulatively to gain a complete understanding of the organization’s risk management processes and make a judgment regarding their effectiveness.
Assessing Organizational Risk Management
Gather Information to Understand the Risk Management Process section is quite good. Then Conduct a Preliminary Risk Assessment is unfortunately bad.
An effective way to perform and document an engagement-level risk assessment is to create a risk matrix listing the relevant risks and then expand the matrix to include measures of significance.
There are better and much more effective ways to identify and prioritise risks than a heatmap. I have a whole article on better alternatives.
The overall objective of an assessment of the organization’s risk management process is typically to provide insight to senior management and the board regarding the maturity of the organization’s risk management and whether it corresponds to their expectations.
This above is the huge misconception. The overall objective of an assessment of the organization’s risk management processes is to determine whether quality decisions are taken and the objectivess are achiavable. Norman Marks calls it the probability of success.
Establish Engagement Scope
The section on scope is probably the most important one, since it shows an indication of what to look out for. In the beginning of the document integration into decision making is mentioned few times and I was wondering whether this section and Appendix E would support the claim. They didn’t. Very basic risk management 1.
- The sufficiency and operating effectiveness of the policies, procedures, and activities that support the risk management process, including alignment with the organization’s risk appetite, stakeholder expectations, and industry standards.
- The effectiveness of governance structures supporting the policies, procedures, and activities related to the risk management process.
- The adequacy of resources dedicated to supporting the risk management process.
- The inclusion of the following in the risk management process
To summarise, following IIA advice in this practice guide, you will still have no idea whether risk management is effective or not, plus you will spend a lot of time asking the wrong questions and reading the irrelevant documents.
