|
The University of the State of New York Office of State Assessment IRT Pattern Scoring of New York State Fourth and Eighth Grade ELA and Mathematics Tests IRT pattern scoring, as opposed to number-correct (NC) scoring, was used in computing student scale scores (SSs) for the New York State assessments. IRT pattern scoring produces scores for individual students that are more accurate than those produced by number-correct scoring. The increase in accuracy is approximately equivalent to a 15 to 20% increase in the number of questions on the test. Number-correct scoring considers only how many questions a student answered correctly in determining his or her score. In contrast, the IRT pattern-scoring method takes into account not only his or her total raw score, but also which questions he or she got right and the psychometric characteristics of these questions. The IRT pattern scoring gains accuracy by considering the fact that the questions are not equal in their characteristics. For example, low-ability students are more likely to guess correctly on some questions than on others, and some questions are more discriminating than others. Two students with exactly the same raw score total will get the same SS in number-correct scoring. However, it is very possible that even though they have the same raw score total, they answered different questions correctly, and that the questions were very different with regard to their characteristics. In such a case, they will likely get different SSs in item-pattern scoring even though they have the same raw score total. For groups of 25 or more students, item-pattern and number-correct scoring methods produce equivalent total scores for the group. For individual students, IRT pattern scoring offers the advantage of making use of the information available in the constellation of questions answered correctly and incorrectly. This offers teachers, children, and parents accurate information about student strengths and weaknesses. 2/1/00 |