This paper addresses the importance of situating explainable AI approaches within human social interactions to improve model transparency. The paper focuses on the concept of “social transparency,” which incorporates the context of those social interactions into explanations of AI systems. Interviews with AI users and practitioners ground the paper’s offering of a conceptual framework for identifying and measuring social transparency in order to improve AI decision making, increasing trust in AI, and nurturing broader values of AI explainability.