Getting model explainability right means everything to AI lending
Zest AI has helped more lenders put AI-driven credit underwriting models into production than anyone on the planet.
We’ve learned — sometimes the hard way — what it takes to get up and running with fully compliant AI credit models that approve more borrowers faster with no added risk. We think (and we’re not alone on this) that every lender will eventually switch from traditional credit scoring to AI-driven lending. The gains are just too big to ignore. Sure, machine learning adds complexity. But it’s the kind of complexity that comes with initially mastering any new technology — and we can help you manage the learning curve. We spent the last half-decade building the technology and automation tools that will help any lender spot borrowers overlooked by legacy credit scores.
But lenders do need to get AI right. So, we want to share what we’ve learned with an industry that’s just beginning to get its arms around the seismic shift. There are some key focuses for enabling machine learning compliance:
- fair lending
- bias
- transparency
- safety and soundness
- data sources and monitoring
There’s a method to the madness
Lenders are legally required to explain why they made their lending decisions, whether the decisions are made automatically using an algorithm or using manual underwriting. Regulators want to ensure lenders aren’t making high-risk loan approvals. And for denials, consumer protection regulators want to ensure that lenders comply with the Equal Credit Opportunity Act and state and federal fair lending laws. For algorithmic decisions, this means that lenders must explain and document how the algorithm works.
A piece of advice: When it comes to decide how you’re going to explain a credit model, be sure you pick a method that matches your decision-making engine. If you outsource your underwriting to an industry score provider, you can (legally) rely on what they tell you. If you use an old-school custom model, you can use old-school explainability methods like drop one or impute median. And if you use ML, you need to switch over to an explainability method that relies on game-theory math.
When lenders rely on the algorithm behind a general industry score produced by a credit reporting agency (“CRA”), they are legally allowed to rely on reason codes provided to them by the CRA. That makes life easy in one sense because the lender has effectively outsourced its underwriting to a third party. But, in another sense, life is much more complicated because industry scores are becoming less accurate every day. Industry scores are widely recognized as racist and unfairly biased against Black people. So, those algorithms and explanations will face increasing regulatory scrutiny as time goes by.
Custom algorithms, especially ML ones, can solve the accuracy and racism problems, but require lenders to produce their own explanations. There are several ways to explain an algorithmic decision — whether it’s made by an ML model or a traditional one. Regulators are generally okay with a method so long as it produces accurate reasons for approvals and denials.
Some lenders try to explain ML models the same way they explain other algorithms. They tend to use one of two seemingly reasonable methods: “drop one” and its cousin “impute median.” The problem with drop one and impute median is that they can only handle single-direction, single-variable relationships (an increase in variable X always increases the score). A machine learning model can have hundreds of credit variables with both positive or negative influences on the score. You need serious math to explain what’s going on inside the ML model.
Serious math for a serious model
So, how does one accurately capture and explain the interaction of the many variables that ML models employ to identify the most important factors influencing the model’s decision?
The answer was originally found though what could be considered the early 1900s version of fantasy football — game theory. Game theory pioneer Lloyd Shapley eventually won the Nobel Prize in Economics because of this work. Mathematicians started trying to quantify how each player on a sports team contributed to the game’s final score, taking into account the number of baskets, touchdowns, or goals the player scored and the player’s assists, passes, and blocks.
Eventually, Shapley’s proofs turned out to be a rock-solid way to explain how ML models make decisions. In the case of an ML model, the “players” are the model variables, the “game” is the model, and the “score” is the model’s output (in credit, this usually represents the probability of defaulting on a loan). That’s why applying game theory math to ML models works so well.
Shapley’s method precisely quantifies the significance of model variables in generating a given score for a given applicant. It also considers variable interactions to know precisely how much each variable contributed to the credit decision. Shapley proved his method is the only reasonable way of assigning credit in games of this sort. We have used Shapley’s approach (and its formal extensions) to explain millions of ML lending decisions for many different kinds of models throughout our history as a company.
Without accurate explainability, lenders don’t have real insight into their lending decisions. That leaves them exposed to financial and compliance risks, often without even knowing it. In contrast, accurate explainability renders the opaque transparent and allows lenders to reap the many benefits of ML underwriting: higher approvals, lower risk, increased fairness and inclusivity.
