Construction of a paper-and-pencil Test of Basic Scientific Literacy based on selected literacy goals recommended by the American Association for the Advancement of Science

We describe the construction of a paper-and-pencil Test of Basic Scientific Literacy (TBSL) specifically designed for high-school leavers entering technikon and university in South Africa. The `true-false-don't know' scientific literacy test-items are based on a pool of 472 items developed previously from selected literacy goals recommended by the AAAS in Science for all Americans. Test-items were pilot-tested on 625 technikon and university students and were included in the 110-item TBSL on the basis of item discrimination, item difficulty and student feedback. The TBSL consists of three subtests based on Jon Miller's three constitutive dimensions of scientific literacy: the nature of science (22 items); science content knowledge (72 items); and the impact of science and technology on society (16 items). About 260 South Africa-based members of various South African professional science and engineering associations participated in setting a performance standard for each of the three dimensions of scientific literacy. These standards were validated using a `contrasting groups' approach. The internal consistency of the individual TBSL subtests, and the reliability of mastery-nonmastery classification decisions based on the performance standard, was found to be about 0.80. The reliability of the overall 110-item TBSL was 0.95.