Op Risk Benchmarking: Inside the G-Sibs

“I often reflect on the fact that RiskMetrics was released in 1992,” says the head of operational risk at a large European bank, referring to the analytics suite JP Morgan published in an attempt to create a standard for market risk. “Thirty years later, despite some good advances in the way we think about operational risk, we still don’t have anything approaching that.”

Op Risk Benchmarking doesn’t aspire to that kind of status. What it does seek to provide are answers to an array of questions that bedevil senior op risk managers: how many people do my peers have in their second-line teams for a particular risk? Do other banks set key risk indicators or controls for hard-to-measure risks? How do they model those risks? How often is it acceptable to breach risk appetite and modelled risk tolerances? Are risk reports creating the ‘right’ kind of actions?

Highlights from each of the five benchmarked risks can be found below. The full article for each risk can be found here.

The project has its origins in Risk.net’s annual Top 10 Op Risks report, which many firms have long used as a form of soft benchmarking, as part of qualitative reporting to boards and business units, and as a means of keeping key stakeholders apprised of what their peers are doing.

After a concerted push, 55 organisations participated in this year’s Top 10 – enough to split the responses into four cohorts, and create separate top five risks for each. Global systemically important banks made up the largest group of respondents at 20 (representing two-thirds of the global population of G-Sibs). The other three cohorts are non-G-Sib banks, asset managers and insurers, and financial market infrastructures.

In another first, the Top 10 survey also included a handful of brief follow-up questions, asking firms whether risks had risen or receded in the previous 12 months, whether they expected that trend to continue, and whether their loss exposure had increased or decreased over the same period. These were used to create Likert scales, giving a useful forward-looking indicator for each risk.

But our ambition went beyond rough gauges of perceived risk – and it was clear subscribers’ did too. Over the course of dozens of hours of interviews, Risk.net discussed the exercise with more than 30 financial firms, first to define the topics where benchmarking would be useful, then to set and focus the questions.

The result was nine clusters of questions, seven covering different aspects of op risk management frameworks, and two focusing on quantification practices. These were: risk appetite setting, including methodologies and appetite breaches; organisational structure, including lines of defence and staffing; key risk indicators; risk controls and control benchmarking; risk reporting; technology, including preferred vendors for each risk; regulation; risk modelling; and scenario analysis.

These nine clusters of questions have been combined with the top five risks chosen by each peer group, creating a unique sweep of measurement and management practices across each cohort’s most pressing concerns.

Firms were free to participate in as many or as few sections as they wished. Some chose to answer every cluster of questions, for all risks; some focused on the two risk quantification sections; others took a risk-by-risk approach, eschewing those they manage separately, particularly regulatory compliance and geopolitical risk. The sample count is therefore shown for each figure in each of the five reports the research produced.

Eleven G-Sibs took part in this first instalment of the exercise, with responses gathered during April and May 2023, yielding more than 1,500 data points. The respondents are roughly one-third Asia-Pacific banks, one-third European, one-third US. Answers were cleaned and aggregated, and used to build up a picture of current practice for each of the top five risks selected by G-Sibs. Follow-up interviews offered insight and supplementary analysis.

Five reports have been produced, for each of the top five risks: cyber risk in the form of IT disruption, which was the top concern for G-Sibs, and of information security, which ranked fifth; execution and process errors; regulatory compliance risk; and geopolitical risk.

The full datasets are available to participants in the exercise. Subscribers have access to selected highlights and commentary – a fairly detailed snapshot. In the months ahead, we will produce reports on the data collected from each of the other cohorts about how they manage the five risks selected by their own peer group.

Risk.net hopes the information will be helpful to a discipline that has grown up rapidly in the past two decades, but often lacks clear standards and best practices. As the head of op risk notes: “This is a discipline where we still need all the external data we can get our hands on.”

If you want to see the full datasets, you’ll need to take part in the next benchmarking round. Message us for details: ORMBenchmarking@risk.net