Summary of Reliability Information of the
Regents Comprehensive Examination in English

Four estimates of the reliability of the Regents Comprehensive Examination in English are summarized below. The first three address interrater reliability of the open-ended scoring, and the fourth addresses, internal consistency of all test questions.

I. Department Review of Regents Comprehensive Examination in English, June 1999

Number of student papers re-read: 2,562

Number of questions read: 10,248

Number of papers with scoring changes: 517

Percent agreement if 4 errors per paper = .80

Percent agreement if 1 error per paper = .95*

II. Range Holistic Scoring Agreement for pretest: .87 — .94**

III. Agreement of Scores — correlation of field test to operational test: .98***

IV. Internal Consistency Reliability:**** .89 (Form A) (Field Test) .89 (Form B)

___________________

* The Department will compile statistics on how many changes were made per paper for a more accurate estimate of agreement.

** ETS

*** Assessment and Evaluation Services — an estimate of interrater agreement

**** Assessment and Evaluation Services

03/08/00