The University of the State of New York

THE STATE EDUCATION DEPARTMENT

Albany, New York 12234

 

 

 

Information Booklet For Administering And Scoring

The Component Retests In English

 

 

GENERAL INFORMATION

The general procedures to be followed in administering Component Retests in English are provided in the publication Directions for Administering and Scoring Component Retests (DET 241). This document is available on the Department’s web site, http://www.emsc.nysed.gov/osa/component.html. Questions about general administration procedures for component retests should be directed to the Office of State Assessment at 518‑474‑8220 or 518‑474-5099. For information about the rating of the Component Retests in English, contact the Office of State Assessment at 518‑474‑5912.

School administrators should photocopy this information booklet and distribute copies to school personnel who will be involved in the administration and scoring of the component retests.

 

ADMINISTERING THE COMPONENT RETESTS

 

Test Description

The Component Retests in English assess student attainment of the learning standards at the commencement level. The component retests are administered in five 50-minute sessions on five successive days. Students are to be allowed a maximum of 50 minutes to complete each session. To achieve the maximum raw score possible on the component retest, the student must complete all five sessions. However, all students who complete at least four of the five sessions will earn an official component retest score. Students who complete fewer than four component retest sessions are considered to have withdrawn from the component retest. Schools may not enter a component retest score in the permanent records of students who sat for fewer than four retest sessions, but their papers may be rated only to provide feedback on their test performance.

Component A covers the informational standard assessed in Session One of the Regents Examination. Modules 1 and 2 each include an extended constructed-response item based on a listening passage. Modules 3, 4, and 5 each include five multiple-choice questions and two short constructed-response items based on an informational piece and a graphic piece.

Component B covers the literary-response and critical-analysis standards that are assessed in Session Two of the Regents Examination. Modules 1, 2, and 3 each include five multiple-choice questions and two short constructed response items based on paired literary passages. Modules 4 and 5 each include one extended constructed-response item based on a critical lens that will be applied to a literary work that the student has read.

 

Test Materials

For each session, each student is to be given the appropriate test booklet for the date and session.  The test booklet includes one or more detachable answer sheets. Students must record their answers to the multiple-choice questions and their short responses or essay on these answer sheets. Each component retest answer sheet also includes a box for recording student scores on questions for that module. The component retest booklets for Module 5 also include spaces for recording the student’s scores for Modules 1 through 5 and for entering the student’s final (total raw) score and score range on the test. (See page 6.)

Each teacher administering Module 1 and Module 2 of the Component A retest should receive the appropriate Teacher Dictation Copy containing the listening passage to be administered as the first part of each of those two test sessions. The Teacher Dictation Copies should be distributed one hour before the scheduled starting time so that teachers will have time to familiarize themselves with the materials before beginning that session of the examination. There are no listening passages for the remaining three modules of the Component A retest or for any of the five modules of the Component B retest.

 

SCORING THE EXAMINATION

No later than three school days after the conclusion of the testing period, rating materials for all of the component retests will be posted on the Department’s web site at:  http://www.emsc.nysed.gov/osa/component.html. Schools will not receive paper copies of the rating materials.

 

The Scoring Keys and Rating Guides

For each module, there will be a separate Scoring Key and/or Rating Guide. Materials for modules with 4‑point extended-response items will contain the scoring rubric along with annotated anchor and practice papers. Modules with multiple-choice items and 2-point short constructed responses will contain a scoring key for the multiple-choice items and scoring rubrics for the 2-point items along with annotated anchor and practice papers. A conversion table for Component A and for Component B will also be provided.

 

Rating the Component Retests

The reliability of the scores is a fundamental concern in the measurement of the student’s achievement. Therefore, the essays (Component A—Modules 1 and 2; Component B—Modules 4 and 5) for the component retests in English must be rated by at least two qualified raters. The 2‑point short responses and multiple-choice questions for the component retests in English may be rated by only one qualified rater. Qualified raters include teachers of English, reading, English as a second language, and special education who know the English curriculum and have received specific training in the scoring of the component retests as part of the turnkey training process that began in March 2001.

In order to ensure reliable scoring, principals of each high school administering the component retests in English must follow the procedures described below:

·           Appoint a scoring coordinator who will manage the training and logistics of the scoring process.

·           Provide task-specific training just prior to scoring.

·           Assign two teachers to rate each essay independently, with a third teacher available to resolve discrepant scores.

Organizing the Rating and Recording

Before student responses can be read and rated, each school must set up a procedure for collecting, arranging, and processing the answer papers and for maintaining records of the component retest results. The procedure used in a particular school must be designed to produce a reliable score for each student and to facilitate maintenance of the school’s records of each student’s score. Some of the suggested rating procedures for the Regents Comprehensive Examination in English might be helpful.

 

Scoring of Multiple-Choice Questions

Modules 3, 4, and 5 of the retest for Component A and Modules 1, 2, and 3 of the retest for Component B contain multiple-choice questions. On the answer sheets for these modules, indicate by means of a check mark each incorrect or omitted answer to multiple-choice questions. Do not place a check mark beside a correct answer. Use only red ink or red pencil. In the boxes provided on the answer sheets, record the number of questions the student answered correctly for that module.

 

Detailed Directions for Training Raters to Score Student Responses

       In training raters to score student responses for each part of the component retests, follow the procedures outlined below:

Introduction to the Task

       The introduction to the task may take place once the administration of all five modules comprising the component retest has concluded and the school has downloaded the rating materials from the Department’s web site.

·           Raters read the task and summarize it.

·           Raters read passage(s) and plan a response to the task.

·           Raters share response plans and summarize expectations for student responses.

Introduction to the Rubric and Anchor Papers

·           Trainer reviews rubric with reference to the task.

·           At this point, raters should also be directed to the additional scoring considerations printed at the bottom of each 4-point rubric.

·           Trainer reviews procedures for assigning holistic scores (i.e., by matching evidence from the response to the language of the rubric and by weighing all qualities equally).

·           Trainer leads review of each anchor paper and commentary.

Practice Scoring Individually

·           Raters score a set of five practice papers individually.

·           Raters should score the five papers independently without looking at the scores and commentaries provided after the five papers.

·           Trainer records scores and leads discussion until raters feel comfortable enough to move on to actual scoring.

Method for Determining the Score for Each Essay

 

Two Ratings:

       1.   Compare the two ratings.

       2.  If the two ratings agree, the student receives that score.

       3.  If the two ratings are contiguous, average the two scores.

       4.  If the two ratings are not contiguous, a third rating is necessary.

 

Three Ratings:

       1.   Compare the three ratings.

       2.  If two of the three ratings agree, the student receives that score.

       3.  If the three ratings are different, the student receives the middle score.

 

Examples:

 

        * If the final score ends in .5, do not round at this point.

Entering Essay Scores on the Record Sheet for Component A

 

The examples below show how students’ scores should be recorded on the Record Sheet.

 

Name	Module 1: Essay Scores				Module 2: Essay Scores
	Rater 1	Rater 2	Rater 3	Resolved Score		Rater 1	Rater 2	Rater 3	Resolved Score
Student A	4	4	—	4		2	4	3	3
Student B	3	1	1	1		1	2	—	1.5

 

 

 

Entering Essay Scores on the Record Sheet for Component B

 

The examples below show how students’ scores should be recorded on the Record Sheet.

 

Name	Module 4: Essay Scores				Module 5: Essay Scores
	Rater 1	Rater 2	Rater 3	Resolved Score		Rater 1	Rater 2	Rater 3	Resolved Score
Student C	2	0	2	2		3	3	—	3
Student D	3	3	—	3		4	3	—	3.5