The University of the State of New York

THE STATE EDUCATION DEPARTMENT

Albany, New York 12234

Information Booklet For Administering And Scoring

The Component Retests In English

GENERAL INFORMATION

The general procedures to be followed in administering Component Retests in English are provided in the publication Directions for Administering and Scoring Component Retests. This document is available on the Department’s web site, http://www.emsc.nysed.gov/osa/component.html. Questions about general administration procedures for component retests should be directed to the Office of State Assessment at 518‑474‑8220 or 518‑474-5902. For information about the rating of the Component Retests in English, contact the Office of State Assessment at 518‑474‑5912.

School administrators should photocopy this information booklet and distribute copies to school personnel who will be involved in the administration and scoring of the component retests.

ADMINISTERING THE COMPONENT RETESTS

Test Description

The Component Retests in English assess student attainment of the learning standards at the commencement level. The component retests are administered in five 50-minute sessions on five successive days. Students are to be allowed a maximum of 50 minutes to complete each session. To achieve the maximum raw score possible on the component retest, the student must complete all five sessions. However, all students who complete at least four of the five sessions will earn an official component retest score. Students who complete fewer than four component retest sessions are considered to have withdrawn from the component retest. Schools may not enter a component retest score in the permanent records of students who sat for fewer than four retest sessions, but their papers may be rated only to provide feedback on their test performance.

Component A covers the informational standard assessed in Session One of the Regents Examination. Modules 1 and 2 each include an extended constructed-response item based on a listening passage. Modules 3, 4, and 5 each include five multiple-choice questions and two short constructed-response items based on an informational piece and a graphic piece.

Component B covers the literary-response and critical-analysis standards that are assessed in Session Two of the Regents Examination. Modules 1, 2, and 3 each include five multiple-choice questions and two short constructed-response items based on paired literary passages. Modules 4 and 5 each include one extended constructed-response item based on a critical lens that will be applied to a literary work that the student has read.

Test Materials

For each session, each student is to be given the appropriate test booklet for the component (A or B), date, and session. The test booklet includes one or more detachable answer sheets. Students must record their answers to the multiple-choice questions and their short responses or essay on these answer sheets. Each component retest answer sheet also includes a box for recording student scores on questions for that module. The component retest booklets for Module 5 also include spaces for recording the student’s scores for Modules 1 through 5 and for entering the student’s final (total raw) score and score range on the test. (See page 7.)

Each teacher administering Module 1 and Module 2 of the Component A retest should receive the appropriate Teacher Dictation Copy containing the listening passage to be administered as the first part of each of those two test sessions. The Teacher Dictation Copies should be distributed one hour before the scheduled starting time so that teachers will have time to familiarize themselves with the materials before beginning that session of the examination. There are no listening passages for the remaining three modules of the Component A retest or for any of the five modules of the Component B retest.

SCORING THE EXAMINATION

On or about May 14, 2007, rating materials for all of the component retests will be posted on the Department’s web site, http://www.emsc.nysed.gov/osa/component.html; paper copies of the rating materials will not be sent to schools. Schools must print sufficient copies of these materials to supply them to each rater.

The Scoring Keys and Rating Guides

For each module, there will be a separate scoring key and/or rating guide. Materials for modules
with 4‑credit extended-response questions will contain the scoring rubric along with annotated anchor and practice papers. Modules with multiple-choice questions and 2-credit short constructed responses will contain a scoring key for the multiple-choice questions and scoring rubrics for the 2-credit items along with annotated anchor and practice papers. A conversion chart for Component A and for Component B will also be provided.

Rating the Component Retests

The reliability of the scores is a fundamental concern in the measurement of the student’s achievement. Therefore, the essays (Component A—Modules 1 and 2; Component B—Modules 4 and 5) for the Component Retests in English must be rated by at least two qualified raters. The 2‑credit short responses and multiple-choice questions for the Component Retests in English may be rated by only one qualified rater. Qualified raters include teachers of English, reading, English as a second language, and special education who know the English curriculum and have received specific training in the scoring of the component retests as part of the turnkey training process that began in March 2001.

In order to ensure reliable scoring, principals of each high school administering the Component Retests in English must follow the procedures described below:

· Appoint a scoring coordinator who will manage the training and logistics of the scoring process.

· Provide task-specific training just prior to scoring.

· Assign two teachers to rate each essay independently, with a third teacher available to resolve discrepant scores.

When the teacher scoring committee completes the scoring process, test scores must be considered final and must be entered onto students’ permanent records.

Principals and other administrative staff in a school or district do not have the authority to set aside the scores arrived at by the teacher scoring committee and rescore student examination papers or to change any scores assigned through the procedures described in this manual and in the scoring materials provided by the Department. Any principal or administrator found to have done so, except in the circumstances described below, will be in violation of Department policy regarding the scoring of State examinations. Teachers and administrators who violate Department policy with respect to scoring State examinations may be subject to disciplinary action in accordance with Sections 3020 and 3020-a of Education Law or to action against their certification pursuant to Part 83 of the Regulations of the Commissioner of Education.

On rare occasions, an administrator may learn that an isolated error occurred in a final score for a student or in recording students’ scores in their permanent records. For example, the final score may have been based on an incorrect summing of the student’s raw scores for parts of the test or from a misreading of the conversion chart. When such errors involve no more than five students’ final scores on any examination and when such errors are detected within four months of the test date, the principal may arrange for the corrected score to be recorded in the student’s permanent record. However, in all such instances, the principal must advise the Office of State Assessment in writing that the student’s score has been corrected. The written notification to the Department must be signed by the principal or superintendent and must include the names of the students whose scores have been corrected, the name of the examination, the students’ original and corrected scores, and a brief explanation of the nature of the scoring error that was corrected.

If an administrator has substantial reason to believe that the teacher scoring committee has failed to accurately score more than five student answer papers on any examination, the administrator must first obtain permission in writing from the Office of State Assessment before arranging for or permitting a rescoring of student papers. The written request to the Office of State Assessment must come from the superintendent of a public school district or the chief administrative officer of a nonpublic or charter school and must include the examination title, date of administration, and number of students whose papers would be subject to such rescoring. This request must also include a statement explaining why the administrator believes that the teacher scoring committee failed to score appropriately and, thus, why he or she believes rescoring the examination papers is necessary. As part of this submission, the school administrator must make clear his or her understanding that such extraordinary re-rating may be carried out only by a full committee of teachers constituted in accordance with the scoring guidelines presented above and fully utilizing the scoring materials for this test provided by the Department.

The Department sometimes finds it necessary to notify schools of a revision to the scoring key and rating guide for an examination. Should this occur after the scoring committee has completed its work, the principal is authorized to have appropriate members of the scoring committee review students’ responses only to the specific question(s) referenced in the notification and to adjust students’ final examination scores when appropriate. Only in such circumstances is the school not required to notify or obtain approval from the Department to correct students’ final examination scores.

Organizing the Rating and Recording

Before student responses can be read and rated, each school must set up a procedure for collecting, arranging, and processing the answer papers and for maintaining records of the component retest results. The procedure used in a particular school must be designed to produce a reliable score for each student and to facilitate maintenance of the school’s records of each student’s score. Some of the suggested rating procedures for the Regents Comprehensive Examination in English might be helpful. (See the Information Booklet for Scoring the Regents Comprehensive Examination in English, which is available on the Department’s web site, http://www.emsc.nysed.gov/osa/hseng.html.)

Scoring of Multiple-Choice Questions

Modules 3, 4, and 5 of the retest for Component A and Modules 1, 2, and 3 of the retest for Component B contain multiple-choice questions. On the answer sheets for these modules, indicate by means of a check mark each incorrect or omitted answer to multiple-choice questions. Do not place a check mark beside a correct answer. Use only red ink or red pencil. In the boxes provided on the answer sheets, record the number of questions the student answered correctly for that module.

Detailed Directions for Training Raters to Score Student Responses

In training raters to score student responses for each part of the component retests, follow the procedures outlined below:

Introduction to the Task

The introduction to the task may take place once the administration of all five modules comprising the component retest has concluded and the school has downloaded the rating materials from the Department’s web site.

· Raters read the task and summarize it.

· Raters read passage(s) and plan a response to the task.

· Raters share response plans and summarize expectations for student responses.

Introduction to the Rubric and Anchor Papers

· Trainer reviews rubric with reference to the task.

· At this point, raters should also be directed to the additional scoring considerations printed at the bottom of each 4-credit rubric.

· Trainer reviews procedures for assigning holistic scores (i.e., by matching evidence from the response to the language of the rubric and by weighing all qualities equally).

· Trainer leads review of each anchor paper and commentary.

Practice Scoring Individually

· Raters score a set of five practice papers individually.

· Raters should score the five papers independently without looking at the scores and commentaries provided after the five papers.

· Trainer records scores and leads discussion until raters feel comfortable enough to move on to actual scoring.

Method for Determining the Score for Each Essay

Two Ratings:

1. Compare the two ratings.

2. If the two ratings agree, the student receives that score.

3. If the two ratings are contiguous, average the two scores.

4. If the two ratings are not contiguous, a third rating is necessary.

Three Ratings:

1. Compare the three ratings.

2. If two of the three ratings agree, the student receives that score.

3. If the three ratings are different, the student receives the middle score.

Examples:

Rater 1	Rater 2	Rater 3	Resolved Score*	Reason
2	2	—	2	Two ratings are the same. Use that score.
2	3	—	2.5	Two ratings are contiguous. Average the two scores.
2	4	4	4	Two ratings are two or more points apart. Third rating is done. Two of the three ratings agree. Use that score.
1	4	3	3	Two ratings are two or more points apart. Third rating is done. Three ratings differ. Use middle score.
0	1	—	0.5	Two ratings are contiguous. Average the two scores.

* If the final score ends in .5, do not round at this point.

Entering Essay Scores on the Record Sheet for Component A

The examples below show how students’ scores should be recorded on the Record Sheet.

Name	Module 1: Essay Scores				Module 2: Essay Scores
	Rater 1	Rater 2	Rater 3	Resolved Score	Rater 1	Rater 2	Rater 3	Resolved Score
Student A	4	4	—	4	2	4	3	3
Student B	3	1	1	1	1	2	—	1.5

Entering Essay Scores on the Record Sheet for Component B

The examples below show how students’ scores should be recorded on the Record Sheet.

Name	Module 1: Essay Scores				Module 2: Essay Scores
	Rater 1	Rater 2	Rater 3	Resolved Score	Rater 1