5 red flags ? when doing a risk assessment RISK-ACADEMY Blog

Risk assessments are probably the most common activities within the risk management profession and there is a very fine line between being a total waste of time and a useful risk management approach. So what are the most common pitfalls, how to avoid them and how to turn risk assessments into a useful decision making tool.

1. Risk assessments disconnected from decisions

One of the most common mistakes is the risk assessment timing. Not how long it takes, which is also often an issue, but when the risk assessments are carried out. Simplifying this a bit, there 4 most common reasons to do a risk assessment:

• someone has a decision to make which is material and involves choices and uncertainty
• optimising a business process
• need input into the budgeting
• need to set up limit monitoring for highly volatile risks

Probably other reasons as well, write in the comments if I forgot an important one. What is definitely not a good enough reason for a risk assessment is “that time of the year”. Companies that carry out risk assessments on an annual or quarterly schedule because it is apparently best practice are completely missing the plot. Risk assessments should be done on a need by need basis. Let’s look deeper at the 4 common reasons for a risk assessment:

A. Someone has a decision to make which is material and involves choices and uncertainty

Some of the most common examples in my work are investment projects and M&A deals. Those are big and risky decisions and testing hypothesis and resilience of the deals to various internal and external risks is a must have. Most investment teams use external and internal due diligence providers, scenarios and sensitivity analysis to stress test their assumptions. This is a good opportunity to run simulations to determine the probability of positive NPV scenarios and the effect various risks could have on forecasted cash flows. Basically whenever there is a choice to make, especially a choice between few risky alternatives, risk assessment is a must have.

And remember, a list of risks and mitigations is never the end game. The information about risks has to lead to something: a choice between alternatives, a reserve, a contingency, risk-adjusted NPV or something similar.

B. Optimising a business process through risk assessment

Risk assessments are often valuable in optimising business processes. The latest work I have been involved is in procurements and includes building a multifactor risk model for supplier accreditation (trying to reduce tax, compliance and bankruptcy risks), pre-qualification (trying to reduce HSE, ESG and performance risks) and risk-based pricing (trying to help compare total cost of ownership vs risk). I am a strong believer that risk is an extra dimensions that allows better informed decision making within business processes.

C. Need input into the budgeting

Another activity that I am currently involved in is building an operational risk profile for each plant and business unit. This means calculating a loss exceedance curve for each operating unit. The curve allows to determine expected losses (that need to be added to the budget, reserved), unexpected losses (risk capital needs to be allocated and sufficient) and tail events (need to be insured if economically viable or retained). The expected losses are also a good indication of how much money is reasonable to spend on mitigations. These risk assessments have multiple useful applications and input into budgeting is one of them.

D. Need to set up limit monitoring for highly volatile risks

Risk assessment is also a powerful monitoring technique. Certain market and credit risks are part of doing business, have significant volatility and need to be monitored on a daily basis against the risk limits. These risk assessments cannot be done manually because of the frequency and math, so need to be automated and added to the management dashboards for monitoring.

2.  Using the methodology that adds error

The second biggest mistake is using the methodology for a risk assessment that adds error to the output. How do you know if your current methodology adds error? You need to back test it. Take old risk assessments and compare to actual risks that happened over time. Both underestimations and overestimations are considered errors. If the errors are more than 50% that means your methodology is worse than chance and should be avoided. Don’t know how to do your own back test, read someone else’s back tests instead. For example common techniques for doing risk assessments: qualitative ratings used to build a heatmap have been proven to add significant error and are considered “worse than useless”. Switch to an alternative if you are using heatmaps now.

In general, back testing risk assessment methodologies is a must have for any risk professional in any industry or maturity.

3. Using the same methodology for various risks

The third mistake is by far the craziest lie risk consultants ever sold us – that we can apply single ERM methodology to assess various risks. This is borderline crazy, as most risks have their own unique risk assessment methodology. I already hinted above that likelihood x consequences based on qualitative scales is not a viable risk assessment technique as it is no better than horoscopes. For example credit risks are estimated using cVaR = EAD*PD*LGD, etc, market risks are estimated using VaR or Expected shortfall if fat tails are present, operational and investment risks can be estimated using CF@risk or VaR, environmental and compliance risks can be better estimated using Expected shortfall since they have super fat tails.

Continue reading to find out how these various can be aggregated during a risk assessment.

4. Incorrectly aggregating risks

Huge mistakes are commonly made when aggregating risks, which leads to significant overestimation or underestimation of risk. Few things to keep in mind when aggregating risks:

• the measure of risk should be sub additive, for example can’t sum orange and red risks and often even VaRs cannot be added, but you can add expected shortfalls which is therefore probably the better measure for risk. This implies that for the risks that you want to be aggregated later your methodology should allow that addition.
• must account for correlation between risks, if risks are not independent then the risk of the sum may be significantly larger or smaller than the sum of the risks.
• consistent confidence level, in order to aggregate risks you need to apply same confidence levels to each of the risks, can’t add 95% cVaR and 99% market ES.
• may not be necessary to aggregate all risk anyway if it doesn’t help the decision at hand.

What else do you need to keep in mind when aggregating risks? Write in the comments.

By the way, most software vendors that claim to have automated risk assessments haven’t got a clue about any of the points above and sell snake oil at best.

5. Outputs of risk assessments are disconnected from budgeting and performance management

The final mistake is thinking that the purpose of the risk assessment is to mitigate identified risk. Too often the outputs of the risk assessment have no direct and immediate impact on the business budget or a decision at hand. Risk assessment should drive action. The action could be escalation in case of the limit breach, stop loss on the trading position, additional reserves or an investment decision. Business should be the ultimate beneficiary of the outputs from the risk assessment.

Let me know what mistakes you came across when doing or reviewing risk assessments.

RISK-ACADEMY offers online courses

+ Buy now

+ Buy now

+ Buy now

Related