Estimating psychometric properties of computerized multistage testing

In recent decades, computerized multistage testing (MST) has been utilized as an alternative testing mode for large-scale testing programs. Although the usage of MST has been discussed extensively, there is little literature that addresses its psychometric properties. The motivation of this dissertation arises from the need to properly estimate psychometric properties under the MST framework. The primary purpose of this dissertation is to propose an integrated set of formulas for estimating psychometric properties for MST, including error variance, reliability, and classification indices. To achieve this goal, existing methods that are used for the traditional P\&P tests are extended to the MST context, if possible; and new methods are proposed based on the characteristics of MST (i.e., routing probability) for the ML and EAP estimators. For the ML estimator, a conventional approach to computing MST test information analytically is also included for estimating the psychometric properties. The proposed approach and the conventional approach are then applied to a single set of MST data, with varied conditions. Further, a simulation approach is employed for the purpose of understanding and comparing the performance of the proposed approach and conventional approach. In the simulation approach, replications of test data are used to obtain results. The main findings of this dissertation were as follows: (1) for the ML estimator, the proposed analytic approach performed better than the conventional approach, indicated by comparing their results with those from the simulation approach; (2) for the EAP estimator, results from using the Bayesian approach approach led to smaller values of marginal error variances and higher values of reliability, compared with the quadrature approach; (3) longer test resulted in decreased error variances and improved reliability; (4) for the EAP Bayesian approach, increasing sample size had a limited impact on the estimation results; (5) for all the approaches, conditional classification consistency and accuracy gradually decreased when the ability parameter was moving further away from the cut scores. All of the estimation formulas provided in this dissertation can be applied directly to MST data in practice.