FinRegLab worked with researchers from the Stanford Graduate School of Business to conduct a ground-breaking evaluation of emerging practices, tools and techniques to explain, document, and govern machine learning models in credit underwriting.
Download our empirical white paper on machine
learning explainability and fairness:
Empirical White PaperThis research focuses on the implications of using machine learning techniques for extending consumer credit by analyzing tools for diagnosing and managing questions about the transparency and fairness of AI-based lending algorithms. Machine learning models have the potential to increase credit access by more accurately identifying applicants who are likely to repay loans and to reduce the number of people given loans that they are unlikely to repay. But because these models can be more complicated to analyze and manage, model transparency has become a critical threshold question for both lenders and regulators.
The research empirically evaluated model diagnostic tools to help lenders address these transparency challenges and manage machine learning underwriting models as required by law. Specifically, the research assesses how these tools can help to generate disclosures explaining why someone was rejected for credit or charged higher prices, identify what factors in the model drive differences in model predictions among different demographic groups, and assess the overall operation of underwriting models for risk management purposes.
The research found some diagnostic tools identified factors that drove different aspects of the credit model’s behavior, although no one tool performed the best across all tasks. The results suggest machine learning tools can reliably explain complex model behavior, but that it is important to interpret tool outputs carefully, particularly given that many features in the model are correlated with each other. While data science techniques are still evolving, the research finds that some explainability tools can provide important information about how ML models operate and that automated debiasing techniques may offer significant improvements in fairness over traditional compliance approaches.
Proprietary model diagnostic tools from the following technology companies
form the basis of our evaluation:
Publications
Explainability & Fairness in Machine Learning for Credit Underwriting: Policy & Empirical Findings Overview (July 2023)
This paper summarizes the machine learning project’s key empirical research findings and discusses the regulatory and public policy implications to be considered with the increasing use of machine learning models and explainability and fairness techniques.
Read MoreMachine Learning Explainability & Fairness: Insights from Consumer Lending
(Updated July 2023)
This empirical white paper assesses the capabilities and limitations of available model diagnostic tools in helping lenders manage machine learning underwriting models. It focuses on the tools’ production of information relevant to adverse action, fair lending, and model risk management requirements.
Read MoreThe Use of Machine Learning for Credit Underwriting: Market & Data Science Context (Sept. 2021)
This report surveys market practice with respect to the use of machine learning underwriting models and provides an overview of the current questions, debates, and regulatory frameworks that are shaping adoption and use.
Read MoreAI FAQs (Series)
To provide a resource for financial services stakeholders, we have prepared FAQs on the use of AI in financial services. Entries highlight technological, strategic, and ethical debates relevant to the use of advanced modeling and analytical techniques in financial services, and many explain concepts relevant to our research on machine learning credit underwriting.
Read AI FAQsContact us
We welcome any comments and suggestions on our work on AI in financial services.
Please contact us using the form below.