The University of the State of New York

THE STATE EDUCATION DEPARTMENT

Albany, New York 12234

 

 

 

Information Booklet For Administering And Scoring

The Component Retests In English

 

 

GENERAL INFORMATION

The general procedures to be followed in administering Component Retests in English are provided in the publication Directions for Administering and Scoring Component Retests (DET 241). This document is available on the Department’s web site, http://www.emsc.nysed.gov/osa/component.html. Questions about general administration procedures for component retests should be directed to the Office of State Assessment at 518‑474‑8220 or 518‑474-5902. For information about the rating of the Component Retests in English, contact the Office of State Assessment at 518‑474‑5902.

School administrators should photocopy this information booklet and distribute copies to school personnel who will be involved in the administration and scoring of the component retests.

 

ADMINISTERING THE COMPONENT RETESTS

 

Test Description

The Component Retests in English assess student attainment of the learning standards at the commencement level. The component retests are administered in five 50-minute sessions on five successive days. Students are to be allowed a maximum of 50 minutes to complete each session. To achieve the maximum raw score possible on the component retest, the student must complete all five sessions. However, all students who complete at least four of the five sessions will earn an official component retest score. Students who complete fewer than four component retest sessions are considered to have withdrawn from the component retest. Schools may not enter a component retest score in the permanent records of students who sat for fewer than four retest sessions, but their papers may be rated only to provide feedback on their test performance.

Component A covers the informational standard assessed in Session One of the Regents Examination. Modules 1 and 2 each include an extended constructed-response item based on a listening passage. Modules 3, 4, and 5 each include five multiple-choice questions and two short constructed-response items based on an informational piece and a graphic piece.

Component B covers the literary-response and critical-analysis standards that are assessed in Session Two of the Regents Examination. Modules 1, 2, and 3 each include five multiple-choice questions and two short constructed-response items based on paired literary passages. Modules 4 and 5 each include one extended constructed-response item based on a critical lens that will be applied to a literary work that the student has read.

 

Test Materials

For each session, each student is to be given the appropriate test booklet for the date and session. The test booklet includes one or more detachable answer sheets. Students must record their answers to the multiple-choice questions and their short responses or essay on these answer sheets. Each component retest answer sheet also includes a box for recording student scores on questions for that module. The component retest booklets for Module 5 also include spaces for recording the student’s scores for Modules 1 through 5 and for entering the student’s final (total raw) score and score range on the test.
(See page 6.)

Each teacher administering Module 1 and Module 2 of the Component A retest should receive the appropriate Teacher Dictation Copy containing the listening passage to be administered as the first part of each of those two test sessions. The Teacher Dictation Copies should be distributed one hour before the scheduled starting time so that teachers will have time to familiarize themselves with the materials before beginning that session of the examination. There are no listening passages for the remaining three modules of the Component A retest or for any of the five modules of the Component B retest.

SCORING THE EXAMINATION

On or about May 22, 2006, rating materials for all of the component retests will be posted on the Department’s web site, http://www.emsc.nysed.gov/osa/component.html; paper copies of the rating materials will not be sent to schools. Schools must print sufficient copies of these materials to supply them to each rater.

The Scoring Keys and Rating Guides

For each module, there will be a separate scoring key and/or rating guide. Materials for modules
with 4‑point extended-response items will contain the scoring rubric along with annotated anchor and practice papers. Modules with multiple-choice items and 2-point short constructed responses will contain a scoring key for the multiple-choice items and scoring rubrics for the 2-point items along with annotated anchor and practice papers. A conversion chart for Component A and for Component B will also be provided.

Rating the Component Retests

The reliability of the scores is a fundamental concern in the measurement of the student’s achievement. Therefore, the essays (Component A—Modules 1 and 2; Component B—Modules 4 and 5) for the Component Retests in English must be rated by at least two qualified raters. The 2‑point short responses and multiple-choice questions for the Component Retests in English may be rated by only one qualified rater. Qualified raters include teachers of English, reading, English as a second language, and special education who know the English curriculum and have received specific training in the scoring of the component retests as part of the turnkey training process that began in March 2001.

In order to ensure reliable scoring, principals of each high school administering the Component Retests in English must follow the procedures described below:

·       Appoint a scoring coordinator who will manage the training and logistics of the scoring process.

·       Provide task-specific training just prior to scoring.

·       Assign two teachers to rate each essay independently, with a third teacher available to resolve discrepant scores.

Organizing the Rating and Recording

Before student responses can be read and rated, each school must set up a procedure
for collecting, arranging, and processing the answer papers and for maintaining records of the
component retest results. The procedure used in a particular school must be designed to produce a
reliable score for each student and to facilitate maintenance of the school’s records of each
student’s score. Some of the suggested rating procedures for the Regents Comprehensive Examination
in English might be helpful. (See the Information Booklet for Administering and Scoring the
Regents Comprehensive Examination in English, which is available on the Department’s web site, http://www.emsc.nysed.gov/osa/hsgen/jan06/det541e.pdf.)

Scoring of Multiple-Choice Questions

Modules 3, 4, and 5 of the retest for Component A and Modules 1, 2, and 3 of the retest for Component B contain multiple-choice questions. On the answer sheets for these modules, indicate by means of a check mark each incorrect or omitted answer to multiple-choice questions. Do not place a check mark beside a correct answer. Use only red ink or red pencil. In the boxes provided on the answer sheets, record the number of questions the student answered correctly for that module.

Detailed Directions for Training Raters to Score Student Responses

       In training raters to score student responses for each part of the component retests, follow the procedures outlined below:

Introduction to the Task

       The introduction to the task may take place once the administration of all five modules comprising the component retest has concluded and the school has downloaded the rating materials from the Department’s web site.

·     Raters read the task and summarize it.

·     Raters read passage(s) and plan a response to the task.

·     Raters share response plans and summarize expectations for student responses.

Introduction to the Rubric and Anchor Papers

·       Trainer reviews rubric with reference to the task.

·       At this point, raters should also be directed to the additional scoring considerations printed at the bottom of each 4-point rubric.

·       Trainer reviews procedures for assigning holistic scores (i.e., by matching evidence from the response to the language of the rubric and by weighing all qualities equally).

·       Trainer leads review of each anchor paper and commentary.

Practice Scoring Individually

·       Raters score a set of five practice papers individually.

·       Raters should score the five papers independently without looking at the scores and commentaries provided after the five papers.

·       Trainer records scores and leads discussion until raters feel comfortable enough to move on to actual scoring.

 

Method for Determining the Score for Each Essay

 

Two Ratings:

1.     Compare the two ratings.

2.     If the two ratings agree, the student receives that score.

3.     If the two ratings are contiguous, average the two scores.

4.     If the two ratings are not contiguous, a third rating is necessary.

Three Ratings:

1.     Compare the three ratings.

2.     If two of the three ratings agree, the student receives that score.

3.     If the three ratings are different, the student receives the middle score.

Examples:

 

            * If the final score ends in .5, do not round at this point.

 

Entering Essay Scores on the Record Sheet for Component A

 

The examples below show how students’ scores should be recorded on the Record Sheet.

 

Name	Module 1: Essay Scores				Module 2: Essay Scores
	Rater 1	Rater 2	Rater 3	Resolved Score	Rater 1	Rater 2	Rater 3	Resolved Score
Student A	4	4	—	4	2	4	3	3
Student B	3	1	1	1	1	2	—	1.5

 

 

 

Entering Essay Scores on the Record Sheet for Component B

 

The examples below show how students’ scores should be recorded on the Record Sheet.

 

Name	Module 1: Essay Scores				Module 2: Essay Scores
	Rater 1	Rater 2	Rater 3	Resolved Score	Rater 1	Rater 2	Rater 3	Resolved Score
Student C	2	0	2	2	3	3	—	3
Student D	3	3	—	3	4	3	—	3.5

 

 

 

Determining the Student’s Final Component Retest Score Range
Unlike the scores earned on Regents Examinations, final scores for the component retests will not be reported on a 0–100 scale. The student’s final score will be designated as one of three possible score ranges: ·       Score range 65 and above A component retest result of score range 65 and above is equivalent to a score of 65 or higher on a Regents Examination and satisfies the State testing requirement for a local or a Regents diploma. ·       Score range 55–64 A component retest result of score range 55–64 is equivalent to a score between 55 and 64 on the corresponding Regents Examination. In schools that have designated 55 as the passing score on the Regents Comprehensive Examination in English for the awarding of a local diploma, a component retest result of score range 55–64 satisfies the State testing requirement for the local diploma. ·       Score range below 55 A component retest result of score range below 55 is equivalent to a score below 55 on the corresponding Regents Examination and does not satisfy the State testing requirement for a local or a Regents diploma.
A table similar to the one at the right appears on the student’s answer sheet for Module 5. In this table, enter the student’s scores for Modules 1 through 5 and add these scores to arrive at the student’s final score. If the sum of the scores for the five modules ends in .5, that score should be rounded up to the nearest whole number at this time. Enter the student’s final score on the answer sheet.	Component A*   Module 1 Score ___________   Module 2 Score ___________   Module 3 Score ___________   Module 4 Score ___________   Module 5 Score ___________   Final Score ______________   Score Range _____________
To determine the student’s final score range, use the conversion chart that accompanies the scoring materials. Locate the student’s final score down the left side of the chart. The score range to the right of the student’s final score is the student’s final score range. Enter the score range (65 and above, 55–64, or below 55) on the answer sheet. The conversion chart will include final scores ranging from 0 to 35 and the score ranges associated with those final scores.
Because the score ranges corresponding to the final raw scores on the conversion chart differ between the Component A retest and the Component B retest, schools must be sure to use the correct conversion chart for the component on which the student tested. The score ranges corresponding to raw scores on the conversion chart also change from one component retest administration to another. Thus, it is also crucial that for each administration, the conversion chart provided in the scoring materials for that administration be used to determine the student’s score range.
* The procedure for determining the student’s score range is the same for Component B except that the school must use the conversion chart provided in the scoring materials for Component B.

 

Appendix

Rubrics

 

 

Component A

(used for 2-point responses that refer only to the text)

 

 

Score Point 2

 

·       presents a well-developed paragraph

·       provides an appropriate explanation

·       supports the explanation with the information from the text

·       uses language that is appropriate

·       may exhibit errors in conventions that do not hinder comprehension

 

Score Point 1

 

·       provides an explanation

or

·       implies an explanation

or

·       has an unclear explanation

AND

·       supports the explanation with partial or overly general information from the text

·       uses language that may be imprecise or inappropriate

·       exhibits errors in conventions that may hinder comprehension

 

Score Point 0

 

·       is off topic, incoherent, a copy of the task and/or text, or blank

·       demonstrates no understanding of the task/text

·       is a personal response

 

 

Component A

(used for 2-point responses that refer to the text and the graphic)

 

Score Point 2

·       presents a well-developed paragraph addressing the task

·       demonstrates basic understanding of the text and graphic

·       supports the explanation with the information from both the text and graphic

·       uses language that is appropriate

·       may exhibit errors in conventions that do not hinder comprehension

 

Score Point 1

·       provides an explanation

or

·       implies an explanation

or

·       has an unclear explanation

AND

·       supports the explanation with partial or overly general information from the text and/or graphic

·       uses language that may be imprecise or inappropriate

·       exhibits errors in conventions that may hinder comprehension

Score Point 0

·       is off topic, incoherent, a copy of the task and/or text, or blank

·       demonstrates no understanding of the task/text

·       is a personal response

 

Component B

(used for 2-point responses that refer only to one text)

 

 

Score Point 2

·       presents a well-developed paragraph

·       provides an appropriate explanation of the literary element or technique chosen

·       supports the explanation with clear and appropriate evidence from the text

·       uses language that is appropriate

·       may exhibit errors in conventions that do not hinder comprehension

 

Score Point 1

·       provides an explanation of the literary element

or

·       implies an explanation of the literary element

or

·       has an unclear explanation of the literary element

AND

·       supports the explanation with partial and/or overly general information from the text

·       uses language that may be imprecise or inappropriate

·       exhibits errors in conventions that may hinder comprehension

 

Score Point 0

·       is off topic, incoherent, a copy of the task/text, or blank

·       demonstrates no understanding of the task

·       is a personal response

 

Note: Since the question specifies choosing one of the authors, if the student responds using both passages, score the portion of the response that would give the student the higher score.

 

Component B

(used for 2-point responses that refer to two texts)

 

 

Score Point 2

·       presents a well-developed paragraph

·       demonstrates a basic understanding of the texts

·       establishes an appropriate controlling idea

·       supports the controlling idea with clear and appropriate details from both texts

·       uses language that is appropriate

·       may exhibit errors in conventions that do not hinder comprehension

 

Score Point 1

·       has a controlling idea

or

·       implies a controlling idea

or

·       has an unclear controlling idea

AND

·       supports the controlling idea with partial and/or overly general information from the texts

·       uses language that may be imprecise or inappropriate

·       exhibits errors in conventions that may hinder comprehension

 

Score Point 0

·       is off topic, incoherent, a copy of the task/text, or blank

·       demonstrates no understanding of the task/text

·       is a personal response

 

Listening and Writing for Information and Understanding (Component A: Modules 1 and 2)