Home / Featured / Deficient Models: Regulator Response Reflects Increased Risk Management Standards

Deficient Models: Regulator Response Reflects Increased Risk Management Standards

By Steven Minsky

Enterprise risk management isn’t relevant to just one industry or even one type of organizational department. Since 2010, when the SEC started requiring boards to report their risk management effectiveness, it has explicitly marked a lack of risk management competence as a major obstacle to healthy operations. Companies are now required to provide evidence of healthy risk identification methods, strategic risk tolerances, and more, at all levels of the organization.

Failed enterprise risk management—and alternatively, risk management that functions as intended but provides insufficient coverage—can cause myriad issues and incidents, ranging anywhere from missed contract renewals to data loss to the failure of key financial models.

Inadequate (deficient) models have recently been drawing more attention from regulators. When banks use deficient models to calculate how much to lend, particularly to residential borrowers, they send a major red flag to regulators like the Basel Committee on Banking Supervision. Internal models are used for more than the calculation of how much to lend a potential borrower. Among other tasks, banks and bank holding companies (BHCs) also use models for budgeting, calculating the probabilities of different events, and analyzing financial statements.

So, while there can be serious consequences if a bank miscalculates how much capital to allocate for lending, such a preventable mistake is a sign that models used for other crucial responsibilities—such as mitigating cyber threats—are also deficient.

If a financial model produces misleading results, the failure can be the result of either of two broad causes:

  1. Coding Errors: The model “broke,” meaning an error was made during the writing or rewriting of the model’s code. Whether the fix is simple or complex, failure to identify existing errors leads to extended but preventable negative outcomes.
  2. Poor Design or Improper Usage: The model functions “properly,” meaning there is nothing wrong with the code itself. In this case, either the design of the model or the manner in which it’s used is at fault. The model needs to be reworked and possibly replaced.

Whatever the cause of model deficiency, the result—reliance on misleading data—is never good. The first scenario is an easier immediate fix, since all it takes is identifying the error and fixing the model’s backend. Spreadsheets are inadequate for managing these issues. Without a systemic approach provided by ERM, it’s difficult for the business to retrace its steps and evaluate every decision that might have been influenced by data produced with that model. Models can have hundreds of inputs, and simply fixing the coding error doesn’t necessarily solve the larger problem.

The second scenario presents more disturbing questions from the start. First, the organization must figure out what makes the model “deficient.” Does the business fail to expand the model as the organization grows and is exposed to more variables? Has a hand-off in change management (from one employee to another) led to inconsistencies? Does a certain part of the model cease functioning, causing it to produce numbers that are incorrect—but not obviously so—again forcing the company to backtrack?

The solution? Banks and other financial institutions need to implement sufficient model risk management. This will drastically reduce the chance that decisions are made based on misleading data, as well as help the company avoid regulatory difficulties.

Flawed Methods for Calculating Borrower Risk Catch Regulator’s Attention

As mentioned above, the Basel Committee is just one of many regulators, both within the U.S. and around the world, turning its eye on financial institutions and their use of internal models. The Bank of England’s Prudential Regulation Authority (PRA), for example, which supervises approximately 1,700 financial institutions (banks, credit unions, investment firms, insurers, etc.) recently published a paper—Residential Mortgage Risk Weights—in which it writes, “in its review of the approaches that firms use, the PRA has identified material deficiencies in risk capture … Where the PRA identifies a material deficiency in risk capture in a firm’s models, the PRA may consider what further steps may be necessary to rectify these deficiencies on a case by case basis.”

The takeaway for financial institutions, both mortgage lenders and otherwise, is twofold:

  • The growing complexity (and necessity) of financial models makes it doubly important to verify both their design and functionality.
  • Failing to fix deficiencies doesn’t just risk incorrect calculations. It also can result in a crackdown by regulators, which would likely include greater restrictions on an organization’s ability to use its own internal models.

Reworking bank models is no easy task. It takes time, industry expertise, and an enterprise risk management program robust enough to identify and prioritize specific risks. As fundamental components of any banking business, models can also cause problems simply by being offline for upgrades or changes.

Despite this, the reality is that the longer deficient models go unaddressed or undiscovered, the greater the damage to the organizations hosting them. Deficiency means possible regulatory trouble and an increased likelihood that costly operating decisions will be made based on misrepresentative data.

Standards for Managing Risk Are Increasing

Recent events related to internal models are the tip of the iceberg when it comes to changing risk management standards. A series of regulations and enhancements have made enterprise risk management (ERM) not just a wise investment, but an obligatory tool. Fortunately, an ERM program can serve as a central hub for all your solutions, including model risk management, vendor management, compliance, and more.

Enterprise risk management serves three fundamental purposes:

  1. A consistent and comprehensive governance approach to identifying models in use and tracking maintenance and testing.
  2. Efficiently identify, assess, and mitigate risks, avoiding costly surprises with a forward-looking approach. Rather than adding to your workload, an effective ERM program consolidates your risk management responsibilities and makes communication between departments straight-forward.
  3. In the event that you are unable to prevent a certain surprise, ERM solutions maintain an electronic trail of your risk management activities, enabling you to demonstrate your organization’s historical due diligence. Many organizations have avoided regulatory penalties thanks to their enterprise risk management records.

Let’s take a look at some of the items that have raised the bar for risk management solutions:

SEC Proxy Disclosure Enhancements

In 2010, the SEC put into effect a series of rules designed “to enhance the information provided to shareholders so they are better able to evaluate the leadership of public companies.” Among other requirements, the rules mandate complete transparency (disclosure) and accountability for “board’s role in risk oversight.”

As a result of this update, boards must either provide evidence of effective risk management (an straight forward process using the electronic trail maintained by ERM systems) or risk being exposed by regulators like the SEC, the Federal Reserve Board of Governors (FRB), the Federal Deposit Insurance Corporation (FDIC), the National Credit Union Administration (NCUA), the Office of the Comptroller of the Currency (OCC), and the Consumer Financial Protection Bureau (CFPB), among others. Additionally, if an incident does occur, boards don’t have the ability to claim innocence; not knowing about a risk is now negligence and is regarded with the same seriousness as fraud, with the penalties being equivalent.

Sometimes, looking outside your industry helps identify trends. Consider two companies: Dwolla (a small e-commerce company) and Volkswagen. Dwolla did not suffer any risk events, but it did maintain subpar data security standards while claiming otherwise. When investigated by the CFPB, the company was fined for risk management negligence.

Volkswagen, on the other hand, has recently been all over headlines for the emissions scandal. The board produced evidence that it was not aware of the devices being used to fool inspectors. In other words, the board said, “We’re innocent.” Even so, because negligence is considered on par with fraud, the board, management, and staff has not escaped criminal indictment. The company has been hit with massive fines and exposed to class-action lawsuits.

These are direct ramifications from the subprime mortgage crisis of 2008-2009. It is only a matter of time until similar scrutiny returns to confirm the implementation of new controls in the mortgage industry.

FINRA Priority Letters

The Financial Industry Regulatory Authority (FINRA)—regulated by the SEC—has authority over organizations like securities firms, brokers-dealers, and mutual funds. FINRA releases a public letter each year detailing its upcoming regulatory and examinations priorities. This letter is a good bellwether for compliant operations.

In the past several years, FINRA has made ERM a top enforcement priority. Other points of focus are most effectively met by implementing a robust risk management solution. The 2014 letter, for example, prioritized due diligence activities (among other items); organizations were and are expected to maintain suitable internal processes for evaluating investors and verifying recommendations for securities purchases. The 2016 letter prioritizes effective management of “employee risk” (risk related to inventive structures and possible avenues for info leakage), cybersecurity, and proper evaluation of outsourced processes (does the organization use an effective combination of due diligence questionnaires and risk assessments as part of its vendor management process?).

Each year, FINRA raises the bar on a variety of risk management initiatives, which by extension raises the bar for ERM; in order to keep up with increasing standards, companies must develop and improve their existing risk management programs and implement enterprise-wide software solutions.

DFAST 2015 (Dodd-Frank Act Stress Test 2015)

The Dodd-Frank Act, enacted in 2010, was the greatest reform to financial regulation in recent United States history. Mainly intended to increase accountability and full disclosure in the financial system, Dodd-Frank included the creation of the Financial Stability Oversight Council (which has authority over financial institutions), a major overhaul of various agency’s oversights, and much more.

DFAST 2015, however put another onus on banks and BHCs. Specifically, DFAST 2015 is designed to ensure that companies can shoulder various financial and economic stresses. In addition to subjecting organizations to an annual, quantitative evaluation (by the Federal Reserve), organizations are also required to perform “company-run stress tests under the same supervisory scenarios and conduct a mid-cycle stress test under company-developed scenarios.”

In other words, DFAST 2015 requires banks to develop and maintain risk management processes that mitigate a host of different problems. Failing to maintain these processes might mean exposure to the financial risks they’re intended to prevent, but it also might mean regulatory penalties.

How Companies Can Mitigate Risks Related to Mortgages, Fraud, and Compliance

As seen by recent statements from the Prudential Regulation Authority (as well as hundreds of enforcement actions), organizations are being held accountable for their internal risk strategies and their ability to demonstrate risk management effectiveness.

Although the main concern with this trend should be keeping ahead of it, it’s also important to consider why events are unfolding the way they are. If regulators are focusing more on risk management, their motivation for doing so should also be a motivation for organizations (independent of the fear of regulatory penalties). The Department of Justice (DOJ) has recently added criminal prosecution and conviction at all levels of the organization as an incentive for companies to accelerate the adoption of enterprise risk management systems.

A recently proposed framework update—Enterprise Risk Management: Aligning Risk with Strategy and Performance—published by the Committee of Sponsoring Organizations of the Treadway Commission (COSO), provides some valuable insights. According to the update, the risk landscape is changing quickly, and organizations must adapt to a volatile business environment that increases in complexity with the emergence of new technologies, processes, and incidents. Additionally, COSO recognizes that “even success can bring with it risk—the risk of not being able to fulfill unexpectedly high demand or the ability to maintain business momentum that has become an expectation, for example.”

There is a common denominator between problems related to deficient models, the Volkswagen scandal, the Dwolla story (not to mention dozens of other events like pipeline spills, data breaches, and fraud), and solution categories at your own company, like vendor management, cybersecurity policies, and regulatory compliance. That common denominator is risk.

The best way to shield your company from harmful surprises – and bolster its value—is to upgrade enterprise-wide, risk-based programs and support them with robust infrastructure.

What does this mean?

Models may only be used in one process, but the impact of decisions based on those models are across departments and activities. Even when a formal ERM program exists, silos are typically using different processes, standards, and criteria, resulting in subjective and incomparable information. This lack of continuity and transparency makes it nearly impossible to prioritize concerns and address them in an efficient manner.

Enterprise-wide means your risk management program receives inputs from all areas of the organization. In many organizations, silos operate independently, which means that even though departments are managing the risks they can—oftentimes quite effectively—they are not properly collecting, recording, and sharing the results. In addition to redundancy it leaves cross-functional risk unmanaged.

The lack of standardization results in systematic issues that remain hidden and unmitigated. Overlapping, common concerns that might require a more widespread mitigation are passed over in favor of point solutions. When your organization is able to collaborate internally, departments avoid redundant activities and more effectively prioritize which risks need to be addressed first. Enterprise-wide systems also embed risk activities to help streamline the processes throughout the organization best suited to identifying risks to everyday activities: the process owners and their managers.

Risk-based means the program is designed to identify root causes, or the drivers of risk. It’s easier to identify a symptom than the root cause. Understanding which processes are the root cause (and should therefore be redesigned) is a major difficulty.

Enterprise risk management (ERM) solutions are the best way to reap the rewards of an enterprise-wide, risk-based approach. ERM systems manage your model risk in a way that is consistent and available to every department. It identifies home-grown “shadow” models used in departments that are not under systematic quality control procedures, ultimately preventing costly errors. It streamlines the identification of root-cause risks and their related impacts across business areas and builds the business case for action. It also focuses resource/personnel engagement on addressing those causes, resulting not only in model compliance, but a more cost-efficient, effective operation, and the elimination of preventable errors.



Steven Minsky is a recognized thought leader in enterprise risk management (ERM). He is the CEO of LogicManager, and the author of the RIMS Risk Maturity Model (RMM). Steven is also a patent author in risk and process management technology and an instructor on many ERM and GRC topics. He can be reached at Steven.Minsky@LogicManager.com.


Be Sociable, Share!
(Visited 131 times, 1 visits today)

Check Also

Avoiding Surprises

Servicing Quality Assurance and Quality Control Monitoring Systems can be Management’s Best Friend By Jim …