Moody’s outlines proxy portfolio backtesting for banks with limited default data

Moody’s outlines proxy portfolio backtesting for banks with limited default data
Backtesting with Proxy Portfolios

Banks with limited internal default histories can still carry out meaningful credit model backtesting by using representative proxy portfolios built from external datasets. The approach is aimed at helping institutions test both risk-ranking performance and probability-of-default calibration across the model life cycle.

Highlights

  • Moody’s proposes a backtesting framework using large external datasets like its Data Alliance, addressing validation gaps for banks with limited internal default data.
  • The proxy portfolio approach enables banks to test models' discriminatory power and level accuracy through measures such as the accuracy ratio and Cumulative Accuracy Profile curve.
  • Federal Reserve SR 26-2 requires banks to conduct outcomes analysis regardless of internal default scarcity, and Moody’s framework allows U.S. banks to meet these robust model performance expectations.

Framework for model validation

As outlined by Moody’s, the proposed framework addresses a persistent validation problem for banks, particularly midsized lenders and institutions whose portfolios have not yet passed through a full credit cycle. In those cases, a lack of internal defaults can limit outcomes analysis, even though model risk management requires comparisons between model outputs and real-world results.

The paper says backtesting is used to answer two core questions, whether a model can consistently rank borrowers by risk and whether its predicted probabilities of default are aligned with observed default rates over time. Federal Reserve guidance, cited as SR 26-2, makes clear that banks must conduct this type of outcomes analysis to assess performance against model objectives and business use.

To address scarce internal default data, the framework calls for building a proxy portfolio from a large external dataset that mirrors the bank’s own risk characteristics. Moody’s says its Data Alliance dataset, which includes millions of financial statements and more than 1 million private-firm defaults, can support the construction of such a commercial lending proxy portfolio.

Risk management implications for lenders

A proxy-based backtesting process is presented as a multi-step analytical exercise designed to reflect a bank’s specific risk exposure. The process begins with defining the institution’s portfolio footprint, including factors such as geography, industry or property type, and borrower or loan size, before using those characteristics to create relevant strata for the proxy portfolio.

Once the proxy portfolio is established, banks can test discriminatory power and level accuracy using standard measures. The paper identifies the accuracy ratio, derived from the Cumulative Accuracy Profile curve, as the primary tool for assessing whether a model rank-orders risk effectively, while level accuracy focuses on whether the magnitude of predicted default probabilities is correct.

Moody’s presents the approach as a practical way for banks to strengthen model validation when internal defaults are too few for reliable statistical testing. The broader implication for the U.S. banking sector is that institutions are still expected to demonstrate robust model performance, even when portfolio history is limited.

Earlier, we discussed Moody’s risk validation used by a European private credit manager to benchmark portfolio credit quality during fundraising.

This material may contain third-party opinions, none of the data and information on this webpage constitutes investment advice according to our Disclaimer. While we adhere to strict Editorial Integrity, this post may contain references to products from our partners.