Comparing t Test, rit Significance Test, and rit Criteria for Item Selection Method: A Simulation Study

Three criteria of items selection have been widely used despite of their limitations without any empirical evidence to support its practice. Current study examined the three criteria to determine which of the three criteria were the best among the others. Those criteria were the item total correlation, its significance by t-test and significance of rit. Simulations were conducted to demonstrate which of the three criteria provided the least errors in both excluding good items and including bad items in the scale. The author manipulate four conditions in conducting simulation study: (a) number of items in a scale; (b) value of rit in population; (c) sample sizes; and (d) criteria in including or excluding items in a scale. The results showed that criteria of rit > .30 provided the least errors of including bad items and excluding good items, particularly when n > 200. The two criteria based on significance test provided the largest errors therefore were not recommended in future practice.