Are risk models irrelevant?

Risk management failed to stop the financial sector walking off the edge of a cliff. Risk models, widely used by the financial industry, also proved completely inadequate. As a result, financial institutions have lost trillions on deals that were calculated by models as low risk. But how much of this was due to bad risk management, inaccurate financial models or poor reading of accurate models?

The trouble with models

The trouble with all risk models is they cannot predict the future. Mathematical models typically analyse past events and make assumptions about the future.

Try and imagine accurately judging the infinite number of possible trajectories that billiard balls could take during a game. It is practically impossible. In the real world, trying to apply a mathematical system to predict the future it becomes equally pointless. Nonetheless, that is what models seek to do. And they are widely relied upon in the financial world to provide a level of certainty about the uncertain.

“At least this crisis provides another set of data for the models to crunch

Vasilis Katsipis, general manager analytics, A M Best Europe

Systemic risk

Over the last couple of decades the global financial system has become so complicated and interrelated that predicting outcomes is very hard. Systemic risk, meaning the spread of instability throughout the entire system, is a new but recognisable feature of the global business environment. Yet most models incorrectly assumed that risks were uncorrelated. ‘Models do not properly understand systemic events, and they do not account for systemic risks,’ says Eddie McLaughlin, managing director of Marsh’s risk consulting practice. This should have been an early warning that models were not keeping pace with developments in the modern world.

But even with perfect models and perfect data, who could have foretold that plummeting real estate prices would lead millions of Americans to default on their mortgages, that these toxic debts would be bundled up, securitised and passed on to financial institutions around the world so the whole infrastructure of the financial market became infected? The systemic nature of the risks made predicting the rollover effects practically impossible.

‘Unknown, unknowns’ are a serious problem for modern risk managers. Only a few people alive today could fathom an economic crisis of the magnitude we are now witnessing. The nearest thing was the Wall Street crash of the 1930s and, although the models are based on historical data, anything believed to be too old was considered irrelevant. The models simply were not built or tested to countenance such a colossal collapse.

“Models do not properly understand systemic events, and they do not account for systemic risks

Eddie Mclaughlin, managing director risk consulting practice, Marsh

‘Sudden fundamental changes can invalidate some of the models basic assumptions,’ adds Christopher Sutherland, head of insurance due diligence in the risk advisory practice at HSBC Insurance Brokers. The models should have been stress tested to consider the outcome of ‘end of the world’ scenarios but the probability was so slim it appeared to be insignificant.

Nassim Nicholas Taleb, a former trader turned philosopher, and author of The Black Swan, has argued that much of the recent turmoil is due to the inability of financial risk models to account for black swans (rare, unexpected but highly significant events), which he says are much more common than we think.

Nevertheless, popular consensus was that models are a good thing. Regulators even encouraged financial institutions to rely on them to judge how much capital they needed to store in reserve. The problem in that case was that simplifications and assumptions had to be made in order to capture the complexity of the financial system. Those who made the models knew that, but how well it was understood by everyone else is up for debate.

One phenomenon that is far too complex to squeeze into a model is human behaviour. Humans are strange and complex beings that cannot be boiled down into mathematical equations. Our decisions are rarely rational. Given the choice of when to pay for a washing machine, most people would decide to pay for it when it is installed, rather than at the checkout. Given the same choice about a holiday, rarely would someone choose to pay for it after they got home. Most people would pay for the holiday up front. Economically this makes little sense, but people would rather relax without having to worry about how to pay later. Models cannot account for irrational human decisions.

“I told the board they ought to slow down

Paul Moore, former head of regulatory risk, HBOS

Another assumption that models had to make was that everyone was following the rules. Unfortunately, Madoff, Satyam, Stanford and many others prove that people engage in activities that are neither responsible, honest nor diligent. Under these conditions there was little hope of the models functioning properly.

Short-termism

AIG’s collapse started the scepticism about models. The financial giant got into trouble because it started selling insurance on the risk of mortgage backed securities defaulting. Models were mistakenly used as a reliable and sufficient quantification of the risks involved in these transactions. Certain people convinced themselves that AIG was not exposed because that is what the models said. In 2006 the company stated that the risk of a payout on its credit default swap insurance products was ‘remote, even in severe recessionary market scenarios.’ But AIG had not pushed its models to assess how market forces could turn the swaps into huge financial liabilities, reported the Wall Street Journal.

Why did AIG plunge head first into a market without properly assessing the risks? Part of the answer probably lies in another human emotion – greed.

“Being introspective about the limitations of the models is
very important

Roger Martin, Dean of the University of Toronto’s Rotman School of Management

‘The models did not drive behaviour, behaviour drove the models,’ says Rob Nieves, an independent risk consultant. He thinks financial models were used to defend risk taking and legitimise business behaviour. Martin Sullivan, AIG’s former chief executive, reportedly told investors that models helped give him ‘a very high level of comfort.’

Besides, there was not enough of an incentive to question what the models said. As long as they continued to suggest the risks were ‘remote’ businesses could feel comfortable ploughing ahead. Risk managers too, may have suffered from model myopia and failed to stress the importance of questioning what the models were saying. ‘Being introspective about the limitations of the models is very important,’ says Roger Martin, Dean of the University of Toronto’s Rotman School of Management.

With the world lulled by a sustained period of growth and stability, few people, politicians, investors and business leaders included, wanted to restrain the financial firms. Even if a robust risk model did raise a red flag, there was far too much pressure on financial firms to deliver big profits. ‘It is tough to stay out of a market when it is making so much money,’ comments William Stovin, president and chief operating officer of Markel International.

In this environment it would have taken a very loud and influential risk manager to influence business strategy, particularly on the basis that a model assigned only a 1% probability that things would end in disaster.

“A bigger problem than the poor models was weak corporate governance and risk reporting lines

Lawrence Dunn, head of risk management business, EMEA, Riskmetrics Group

Short-termism was a major hallmark of the current crisis and may continue to be a characteristic of future crises as long as chief executives are rewarded on the basis of quarterly results.

Risk manager = party pooper

To be really effective risk managers need to be confident in their abilities and have a direct reporting line to the top management. No one wants to be the one to say the party is over. But this burden falls on the risk manager’s shoulders.

A sad lesson from the crisis, however, is that all too easy for the risk department’s calls for caution to be ignored or silenced. A former senior risk manager at HBOS – the British bank that had to be rescued by a government backed takeover – said he warned his bosses that the bank was swimming without any shorts, but his concerns were ignored and eventually led to his dismissal. ‘Being an internal risk and compliance manager at the time felt a bit like being a man in a rowing boat trying to slow down an oil tanker,’ said Paul Moore in written evidence presented to a government inquiry. ‘I certainly knew that the bank was going too fast, had a cultural indisposition to challenge and was a serious risk to financial stability and consumer protection,’ he said. ‘I told the board they ought to slow down.’

“It is tough to stay out of a market when it is making so much money

William Stovin, president and chief operating officer, Markel International

‘A bigger problem than the poor models was weak corporate governance and risk reporting lines,’ says Lawrence Dunn, head of risk management business, EMEA, at Riskmetrics Group. In many organisations, risk managers have no real voice. Moore believes the failure of the banks was due to a breakdown of corporate governance checks and balances. He thinks a direct line to a truly independent non-executive director and to external auditor risk managers could have prevented the crisis.

In defence of models

Just as some people in the financial system absolved responsibility for assessing risks to the rating agencies, models allowed people to become lazy. At the risk of regurgitating a massive cliché, models are just one tool. They are a necessary starting point, but they can not be relied upon. Truly understanding risk requires a balance of quantitative and qualitative risk analysis. ‘Models are not a substitute for good judgment,’ notes McLaughlin.

Modelling is not a new thing and it is not likely to go away just because it failed, to spot the financial crisis. ‘The limitations of models have always been known but have been intentionally ignored,’ says David Hancock, an independent risk consultant.

In a way the models became a substitute for human judgment. Excessive reliance on them, combined with an inadequate understanding of their underlying assumptions, a lack of attention to extreme scenarios and plain old fashioned greed, drove acceptance of risk beyond what many organisations could sustain.

Hopefully, lessons will be learnt. But inevitably, as long as incentives remain aligned with risk taking, the business world is set for many more crises. At least this crisis provides another set of data for the models to crunch, notes Vasilis Katsipis, general manager analytics, A M Best Europe.

The most important lesson is that risk managers be able to challenge authority. For some organisations that will require a major overhaul of the corporate governance framework.

Non-executive directors need to question what risks are being taken. The risk department needs to learn to communicate complex information clearly. It is going to be challenging, but senior managers should be more willing to listen, now that they have seen what can happen if they do not.