H&G
The University of the State of New York
THE STATE EDUCATION DEPARTMENT
Albany,
New York 12234
The general procedures to be
followed in administering Regents examinations are provided in the publications
Regents Examinations, Regents Competency
Tests, and Proficiency Examinations: School Administrator’s Manual, 2001
Edition, and Directions for
Administering and Scoring Regents Examinations. These publications may be
accessed on the Department’s web site at http://www.emsc.nysed.gov/osa/hsinfogen/hsinfogenarch/sam2001.pdf and at http://www.emsc.nysed.gov/osa/hsgen.html, respectively. Questions about general
administration procedures for Regents examinations should be directed to the
Office of State Assessment at 518-474-8220 or
518-474-5099. For information about the rating of the Regents Examinations in
Global History and Geography and United States History and Government, contact
JoAnn Larson of the Office of Curriculum, Instruction and Instructional
Technology at 518-474-5922 or Gary Warren, Donna Merlau, or Greg Wilsey of the
Office of State Assessment at 518-474-5900.
All school personnel who will be involved in the administration and scoring of these examinations must have a copy of this information booklet. This information booklet may be photocopied; multiple copies of both the rating sheet and the record sheet will be needed for all teachers participating in the scoring of either social studies Regents examination.
The
Regents Examinations in Global History and Geography and in United States
History and Government assess student achievement of the New York State
Learning Standards in social studies at the commencement level. Students are to
be allowed a maximum of three hours to complete each examination.
Each
examination has three parts. Students are to answer all questions in all parts.
·
Part I consists of 50 multiple-choice questions, for
which the student is to select the correct answer from among the four choices
given.
·
Part II contains one thematic essay.
·
Part III A contains several scaffold (open-ended)
questions based on specific documents.
·
Part III B contains one document-based question (DBQ)
essay.
Students
should use black or dark blue ink to write their answers to the essay questions
and the scaffold questions. If students are using a machine-scorable answer
sheet provided by the school for Part I, they will also need a No. 2 pencil.
Each student should be given:
·
An examination booklet with a detachable answer sheet
printed as the last page. Students are to record their answers to the
multiple-choice questions on this answer sheet. The answer sheet includes a
space for recording student scores for Parts I through III and for recording
the student’s final score on the examination. (See page 8.)
Note:
Schools that choose to use machine-scorable answer sheets for Part I should
refer to Regents Examinations, Regents
Competency Tests, and Proficiency Examinations: School Administrator’s Manual,
2001 Edition, for guidelines concerning the format and scoring of the
answer sheets.
· An essay booklet in which students are to write their final thematic and document-based essays. Essay booklets are provided to schools by the Department.
· Scrap paper for planning essays. Scrap paper is provided by the school.
For purposes of a Regents-endorsed diploma,
a score of 65 shall be considered passing. For students who first started Grade
9 in the 1998-99 through 2004-05 school years, scores of 55-64, as determined
by the school, may be considered passing for the awarding of a local diploma.
The
Scoring Key and Rating Guide
The
scoring key and rating guide contains:
·
Correct answers to the multiple-choice questions.
·
Specific scoring rubrics for both the thematic (Part II)
and DBQ essays (Part III B) and the DBQ scaffold (open-ended) questions (Part
III A).
·
Prescored anchor papers at each essay score level, with
commentary explaining why a particular student paper was awarded that specific
score.
·
Prescored practice papers.
Rating
the Examination
The
reliability of the scores is a fundamental concern in the measurement of the
student’s achievement. Therefore, each essay must be scored by at least two
qualified teachers. The short-answer document-based scaffold questions need
only be scored by one qualified teacher. Qualified raters include teachers of
Grades 7-12 social studies and special education teachers who are knowledgeable
about the Global History and Geography or United States History and Government
curriculum. Raters should have previously received some school-level,
district-level, or regional training on scoring social studies essays or
scaffold (open-ended) questions as part of the turnkey training process.
It is
recommended that schools with a small number of qualified social studies raters
form a consortium of teachers to score as a group the answer papers from
several schools.
In order to ensure reliable scoring, the
principal of each high school administering the social studies Regents
examinations must appoint a scoring coordinator who will:
·
Manage the training and logistics of the scoring process.
·
Provide task-specific training, including review of the
rating guide just prior to scoring.
·
Assign two teachers to rate each essay response
independently, with a third teacher available to resolve discrepant scores. (A
discrepant score is one that varies by more than one point on a 5‑point
rubric.) Only one rater is needed for the scaffold questions. If staffing is
sufficient, separate teams of teachers should rate the Part II thematic essay,
the Part III A scaffold questions, and the Part III B essay.
Every effort should be
made to avoid having a teacher rate his or her own students’ responses. When
this is not feasible, a teacher should score no more than one part of his or
her students’ papers (i.e., a thematic essay, the scaffold questions, or a DBQ
essay).
Organizing
the Rating and Recording
Before student responses can be read and rated, each school must set up a
procedure for collecting, arranging, and processing the answer papers and for
maintaining records of the examination results. The procedure used in a
particular school should be designed to produce a reliable score for each
student and to facilitate maintenance of the school’s records of each student’s
score. A suggested procedure for managing the mechanics of the rating process
is described on pages 4 and 5.
Indicate by means of a
check mark each incorrect or omitted answer to multiple-choice questions on the
designated answer sheet. Do not place a check mark beside a correct answer. Use
only red ink or red pencil. In the appropriate space on the student’s answer
sheet, record the number of multiple-choice questions the student answered
correctly.
In training raters to score student
responses for Part II and Part III of the examination, follow the procedures
outlined below:
1. Introduction
to the Task
The
introduction to the task may take place once the administration of the
examination has begun. However, the actual scoring key and rating guide
for this examination may not be
removed from the shrink-wrapped package of scoring keys for use by raters until
after the Uniform Statewide Admission Deadline has passed.
a. Raters
read the task.
b. Raters identify answers to the task.
c. Raters discuss possible answers and
summarize expectations for student responses.
2. Introduction
to the Specific Rubric and Anchor Papers
The
introduction to the specific rubric and anchor papers may take place once the
Uniform Statewide Admission Deadline has passed.
a. Trainer
leads review of specific rubric with reference to the task.
b. Trainer
leads discussion of procedures for assigning holistic scores (i.e., by matching
evidence from the response to the rubric).
c. Trainer leads review of each anchor paper
and commentary.
3. Practice
Scoring Individually
a. Raters
score the practice papers independently without looking at the scores and
commentaries provided after the papers.
b. Trainer records scores and leads discussion
of scoring criteria until raters feel confident enough to move on to actual
scoring.
c. If additional practice is required to reach scoring consensus, trainer may use a sample of student answer papers from the current administration of the examination.
Suggested
Rating Procedure
The
following procedure is recommended for managing the mechanics of the rating
process. A copy of the rating sheet and the record sheet are included in the
Appendix. You may photocopy as many copies as needed.
1.
The person assigned as the coordinator of the rating
process, or other designated representative(s), will be responsible for
coordinating the movement of papers, calculating a final score for each
student’s essays, recording that information on the student’s Part I answer
sheet, and determining the student’s final score for the examination.
2.
Set aside one room as a central rating room for
collecting, sorting, circulating, and storing answer sheets/essay booklets and
for preparing and maintaining records for these examinations.
3.
Provide a suitable location for rating of essays.
4.
Allow time to provide training for scoring the specific
task for all raters immediately before the rating of the students’ responses
(about 2 hours per essay and about 30-45 minutes per document). It is strongly
recommended that teachers be trained on one document and score those responses,
then be trained and score the responses to the next document, etc.
5.
Provide adequate time for rating (3-5 minutes per
response for each essay, 1 minute per response for each scaffold question scored
0-2, and ½ minute per response for each scaffold question scored 0-1).
For
Part II and Part III B, continue with these procedures:
6.
Each essay must
be scored by at least two qualified teachers. For each essay question, divide
raters into two-person teams. Designate one team member as Rater 1 and the
other as Rater 2. After the examination has been administered, either keep
the essay booklets together and shift them between raters or separate the
students’ essay booklets into Part II and Part III B. If the essay answer booklet is separated, be sure to verify that the
student has entered his or her name and the school name on the page where Part
III (Document-Based Question) begins before separating the parts. After
separating the essays, staple together all pages of the student’s Part II
response and staple together all pages of the student’s Part III B response.
7.
Arrange the essay responses for each part according to a
sequence, using whatever order is most convenient for your school, e.g., class
period, alphabetical, or local identification number. Beginning with the first
paper in the sequence, enter each student’s name on a copy of the record sheet.
(Master for duplicating appears in the Appendix.)
8.
Divide each group of essays into bundles of 25-30
papers.
9.
Prepare a rating sheet for each bundle. (See sample
rating sheet in the Appendix.) After recording the students’ names on the
rating sheet, photocopy the rating sheet. Each rater will need a separate
rating sheet for each bundle of 25 essay papers he or she rates. The second
rater must not be aware of the score assigned by the previous rater.
10. Distribute
the bundles of essay papers to the rating teams, making sure that each rating
team receives two rating sheets for each bundle of papers. Each rater on a team
should rate one of the bundles and record his or her ratings on one of the
rating sheets. The two raters should then exchange bundles. The second rater
should only record his or her scores on the second rating sheet. No scores or corrections should be indicated
on the essay papers.
11. After
each team has completed rating a bundle, the team should return those answer
papers to the central rating room. Remove the rating sheets completed by each
rater from the bundles and enter the scores on the record sheet. Make sure
there are two independent ratings for each response. Enter the resolved scores
in the appropriate columns on the record sheet.
12. Review
the two scores for each student to determine if the student’s scores for that
essay are discrepant, i.e., a difference of more than one point between the two
scores. Separate the students’ responses with discrepant scores and make
another bundle. Prepare a separate rating sheet for those discrepant papers.
List the names of the students on a new rating sheet and attach the sheet to
the corresponding bundle of student responses. Assign each of these bundles to
a rater to obtain a third independent rating of the students’ responses. Make
sure that the third rater is not one of the original two raters of that task
and that the third rater has undergone the training for that task.
13.
After the necessary third ratings have been obtained,
remove the rating sheets from the
bundles of student responses and determine the resolved scores by using the
method for resolving discrepant scores described on the next page. Enter the
resolved scores in the appropriate columns on the record sheet.
14.
Transfer the resolved scores to the appropriate spaces
on the students’ Part I answer sheets.
15.
All rating sheets and record sheets used in scoring the
social studies Regents examination must be kept for at least one year by the
school where they were administered.
For Part III A:
1.
Follow a similar procedure for processing the papers.
2.
The short-answer (open-ended) questions need only be
scored by one qualified teacher.
3.
The scores for each scaffold question may be recorded in
the student’s examination booklet.
4.
Record the total Part III A score in the space provided
on the student’s Part I answer sheet.
Method for Determining the Score for Each Essay
Two
Ratings:
1. Compare
the two ratings.
2. If
the two ratings agree, the student receives that score.
3. If
the two ratings are contiguous, average the two scores.
4. If
the two ratings are not contiguous, a third rating is necessary.
Three
Ratings:
1. Compare
the three ratings.
2. If
two of the three ratings agree, the student receives that score.
3. If
the three ratings are different, the student receives the middle score.
Examples:
|
Rater
1 |
Rater
2 |
Rater 3
|
Resolved
Score*
|
Reason
|
|
2 |
2 |
— |
2 |
Two
ratings agree. Use that score. |
|
2 |
3 |
— |
2.5 |
Two
ratings are contiguous. Average the two scores. |
|
2 |
4 |
4 |
4 |
Two
ratings are two or more points apart. Third rating is done. Two of the three
ratings agree. Use that score. |
|
2 |
4 |
3 |
3 |
Two
ratings are two or more points apart. Third rating is done. Three ratings
differ. Use the middle score. |
|
0 |
1 |
— |
0.5 |
Two
ratings are contiguous. Average the two scores. |
* If the final score ends in .5, do not
round at this point.
Entering Essay Scores on the Record Sheet
The examples below show how students’ scores should be recorded on the record sheet.
|
Student’s Name |
Part II Essay Scores |
Part III B Essay Scores |
||||||
|
Rater 1 |
Rater 2 |
Rater 3 |
Resolved Score |
Rater | ||||