A Theoretical Assessment Ecosystem for a Digital-First Assessment - The Duolingo English Test

The Duolingo English Test is a groundbreaking, digital­first, computer­adaptive measure of English language proficiency for communication and use in English­medium settings. The test measures four key English language proficiency constructs: Speaking, Writing, Reading, and Listening (SWRL), and is aligned with the Common European Framework of Reference for Languages (CEFR) proficiency levels and descriptors. As a digital­first assessment, the test uses “human­in­the­loop AI” from end to end for test security, automated item generation, and scoring of test­taker responses. This paper presents a novel theoretical assessment ecosystem for the Duolingo English Test. It is a theoretical representation of language assessment design, measurement, and test security processes, as well as the test­taker experience factors that contribute to the test validity argument and test impact. The test validity argument is constructed with a digitally­informed chain of inferences that addresses digital affordances applied to the test. The ecosystem is composed of an integrated set of complex frameworks: (1) the Language Assessment Design Framework, (2) the Expanded Evidence­Centered Design Framework, (3) the Computational Psychometrics Framework, and (4) the Test Security Framework. Test­taker experience (TTX) is a test priority throughout the test­taking pipeline, such as low cost, anytime/anywhere, and shorter testing time. The test’s expected impact is aligned with Duolingo’s social mission to lower barriers to education access and offer a secure and delightful test experience, while providing a valid, fair, and reliable test score. The ecosystem leverages principles from assessment theory, computational psychometrics, design, data science, language assessment theory, NLP/AI, and test security.