Cross-national measurement invariance of TIMSS 2023 digital and environmental scales: an alignment optimization approach

Abstract Background TIMSS 2023 introduced three student-report scales—Digital Self-Efficacy, Environmental Attitudes, and Environmental Behaviors—to assess competencies relevant to 21st-century learning. Valid cross-national comparisons require that these scales function equivalently across education systems. This study evaluated their psychometric properties and measurement invariance to assess their suitability for cross-national use. Methods Data from Grade 8 students across 44 education systems were analyzed. Multiple-group confirmatory factor analysis (MGCFA) tested configural, metric, and scalar invariance, followed by alignment optimization to evaluate approximate invariance and support latent mean comparisons. Models were estimated using robust maximum likelihood with full information maximum likelihood for missing data. Results Configural invariance was supported for all scales, but evidence for metric invariance varied across scales and scalar invariance was not achieved. Alignment optimization indicated strong approximate invariance for Digital Self-Efficacy (10.6% non-invariance), acceptable invariance for Environmental Behaviors (21.7%), and more limited invariance for Environmental Attitudes (28.4%). High R² values (≥ 0.957) suggested that most variation reflected substantive cross-national differences rather than measurement bias. Latent mean comparisons revealed distinct patterns, with Digital Self-Efficacy generally higher in several European and East Asian systems, while environmental measures showed relatively higher levels in some Central Asian and Middle Eastern systems. Conclusions The findings provide initial cross-national support for the TIMSS 2023 scales and highlight the value of alignment optimization for large-scale assessments. Digital Self-Efficacy showed relatively strong comparability, whereas environmental measures were more context-sensitive. Results support cautious interpretation of cross-system differences and point to the need for continued scale refinement and external validation.