Underspecification Presents Challenges for Credibility in Modern Machine Learning

This paper explores the problem of “underspecification” – a statistical phenomenon that occurs when an observed issue may have several possible causes, not all of which are accounted for in the model. The team of authors from Google examined case studies in computer vision, medical imaging, natural language processing, and medical genomics, and found variation in model performance based on underspecification problems using a variety of ML pipelines. As a result, training processes that can produce sound models often result in poor models, and the difference between the two will not be apparent until the model is in use and has to generalize to non-training data. Based on these findings, the authors point to greater rigor in specifying model requirements and stress testing models before they are approved for use.