Publications

Machine Learning Explainability & Fairness: Insights from Consumer Lending


Empirical White Paper (Updated July 2023)

Report Summary


This empirical white paper is part of a broader research project on the explainability and fairness of machine learning in credit underwriting. The empirical research was conducted in collaboration with Professors Laura Blattner and Jann Spiess at the Stanford Graduate School of Business.

This empirical white paper is part of a broader research project on the explainability and fairness of machine learning in credit underwriting. The empirical research was conducted in collaboration with Professors Laura Blattner and Jann Spiess at the Stanford Graduate School of Business.

The evaluation analyzes model diagnostic tools from seven technology companies–ArthurH2O.aiFiddlerRelationalAISolasAIStratyfy, and Zest AI–as well as several open-source tools.

This white paper is an update of research published in April 2022.

Acknowledgments


Principal Investigators and Other Contributors:

Laura Blattner

Co-principal investigator, is a former Assistant Professor of Finance at the Stanford Graduate School of Business. Laura earned her Ph.D. at Harvard University. She also holds a B.A. in Philosophy, Politics, and Economics and an M.Phil. in Economics from the University of Oxford. At Stanford, Laura teaches an MBA class on Financial Technology (FinTech). Laura’s research focuses on the governance and regulation of algorithmic credit underwriting.

P-R Stark

served as FinRegLab’s first Director of Machine Learning Research. In that role, she led the organization’s efforts to develop and execute policy-relevant research on the use of AI in financial services. She is an experienced advisor to financial institutions, helping firms respond to regulatory inquiries and manage adoption of new technologies, among other things. P-R holds an A.B. in Classics from Princeton University, an M.A. (Oxon) in Philosophy, Politics, and Economics from the University of Oxford, and a J.D. from Harvard University.

Jann L. Spiess

Assistant Professor of Operations, Information & Technology

is an Assistant Professor of Operations, Information & Technology at Stanford University’s Graduate School of Business. He is an econometrician working on machine learning and causal inference. Jann is particularly interested in developing methods for transparent, robust, and replicable inferences from complex data.

Contributing Authors:

Sarah Davies

Advisor
FinRegLab

Zishun Zhao

Senior Data Scientist
FinRegLab

Data Science Team:

FinRegLab would also like to recognize the presenters and members of our project Advisory Board who contributed to productive discussion of the development of the design, execution, and interpretation of this research. The Advisory Board consists of subject matter experts from computer science, economics, financial services, and regulatory backgrounds and includes representatives from approximately 30 major institutions including bank and nonbank financial institutions, technology firms, advocacy and civil society organizations, and academic institutions. State and federal regulators participated as observers in Advisory Board meetings.

We would also like to thank the following individuals who provided valuable feedback on this report:

Dr. Marsha Courchane

Vice President, Practice Leader of Financial Economics
Charles River Associates

Dr. Adam Gailey

Principal, Financial Economics Practice
Charles River Associates

Steve Dickerson

Discover Financial Services

Raghu Kulkarni

Discover Financial Services

Kate Prochaska

Discover Financial Services

Patrick Hall

Principal Scientist
bnh.ai

Stephen Hayes

Partner
Relman Colfax PLLC

Eric Sublett

Partner
Relman Colfax PLLC

Scott Lundberg

Microsoft Research

Michael Umlauf

Senior Vice President, Data Science & Analytics
TransUnion

Gene Volcheck

TransUnion

We would like to acknowledge the FinRegLab team who worked on convenings and reports related to this project:

Natalia Bailey

Research Manager

Alex Bloomfield

Communications and Development Manager

Kelly Thompson Cochran

Deputy Director and Chief Program Officer
FinRegLab

Colin Foos

FinRegLab

Gillous Harris

Research Analyst
FinRegLab

Kerrigan Molland

Senior Finance & Business Operations Manager
FinRegLab

With Support From


Mastercard Center for Inclusive Growth

The Mastercard Center for Inclusive Growth advances equitable and sustainable economic growth and financial inclusion around the world. The Center leverages the company’s core assets and competencies, including data insights, expertise, and technology, while administering the philanthropic Mastercard Impact Fund, to produce independent research, scale global programs and empower a community of thinkers, leaders, and doers on the front lines of inclusive growth. The Center has provided funding to support this research.

JPMorgan Chase & Co.

(NYSE: JPM) is a leading financial services firm based in the United States of America (“U.S.”), with operations worldwide. JPMorgan Chase had $3.9 trillion in assets and $313 billion in stockholders’ equity as of June 30, 2023. The Firm is a leader in investment banking, financial services for consumers and small businesses, commercial banking, financial transaction processing and asset management. Under the J.P. Morgan and Chase brands, the Firm serves millions of customers in the U.S., and many of the world’s most prominent corporate, institutional and government clients globally. Information about JPMorgan Chase & Co. is available at www.jpmorganchase.com.

Flourish Ventures

Flourish, a venture of the Omidyar Group, has provided operating support to FinRegLab since its inception. Flourish is an evergreen fund investing in entrepreneurs whose innovations help people achieve financial health and prosperity. Established in 2019, Flourish is funded by Pam and Pierre Omidyar. Pierre is the founder of eBay. Managed by a global team, Flourish makes impact-oriented investments in challenger banks, personal finance, insurtech, regtech, and other technologies that empower people and foster a fairer, more inclusive economy.

Related Publications

  • Explainability and Fairness in Machine Learning for Credit Underwriting

    FinRegLab worked with a team of researchers from the Stanford Graduate School of Business to evaluate the explainability and fairness of machine learning for credit underwriting. We focused on measuring the ability of currently available model diagnostic tools to provide information about the performance and capabilities of machine learning underwriting models. This research helps stakeholders… Learn More


Events


About FinregLab

FinRegLab is an independent, nonprofit organization that conducts research and experiments with new technologies and data to drive the financial sector toward a responsible and inclusive marketplace. The organization also facilitates discourse across the financial ecosystem to inform public policy and market practices. To receive periodic updates on the latest research, subscribe to FRL’s newsletter and visit www.finreglab.org. Follow FinRegLab on LinkedIn and Twitter (X).

FinRegLab.org | 1701 K Street Northwest, Suite 1150, Washington, DC 20006