Test Review: For additional information, call or write: John Quinn (Music) >> Rochelle: Good evening. Welcome to "New York Learns: Test Review." I'm Rochelle Cassella, your moderator. Well, it's ten weeks away and counting to the blastoff of the revised Regents comprehensive exam in English. We know that teachers have been preparing students for months, but tonight we're going to review some tips that teachers can use in the classroom for final preparation. First, however, we'd like to bring you up to date on issues related to scoring exams. Before we do so, however, I would like to remind teachers watching that this is your program, really your last opportunity to ask questions related to the exam directly of Jackie Marino, whether it's logistics about scoring day or if you'd like to share some tips about getting students ready for test days. You can do so by calling us directly to talk with us. The toll-free telephone number at our studio at Hudson Valley Community College in Troy is 888-313-4822. You can also fax us if you're shy and don't want to talk. The number is 518-266-8136. But don't be shy because we would like to hear from you. The lines are open; we're waiting for your questions. We'll take them at any point during the program. Now, with the latest and greatest information about scoring day and scoring information, it comes to us from Jacqueline Marino, Associate in English Language Arts from the State Education Department. Jackie, you have to tell us about what's going to be coming in the mail? >> Jackie: Well, we can probably start with a time line of what to expect and when to expect it. I should mention first, the "Testing 1-2-3" booklet, which includes a lot of information, some of which has appeared elsewhere, but some of it is new as well and can help teachers and administrators think about the demands of the tests and how to relate those to their planning for the coming year. >> Rochelle: It's this booklet that many of you have received in the mail, or if you haven't, or would like additional copies, we'll have some information towards the end of the program on where you can go for that. Okay? >> Jackie: And then very soon -- I have been told this week -- we'll be sending out a memo under Jim Kadamus' name called "The Guidance on the 1998-99 State Assessment System," and that, as I said, will be going out -- there will be copies for all teachers in the state. They'll be coming in bundles to individual principals. So the field memo is on its way. And that provides an update on information on all of the new assessments. And then some time in the next month, by mid-May, we hope, you'll be seeing an informational booklet on administering and scoring the Regents exam, and that will go to principals and more about that in a bit when I tell you what will be in that booklet. And then during test week, people will be getting test materials and scoring materials in their secure delivery, depending on whether they're on a daily delivery or not daily delivery. They'll be coming together during test week. And then task-specific training also needs to go on the day that the teachers begin to score the exams. No matter how much time -- and we hope people have been spending a lot of time preparing for scoring the exams, but they haven't been working with the tasks that students will get on that day, of course, so it's necessary to spend a good chunk of time going over the texts and the tasks and relating those to the scoring rubrics and using the materials we send to them. The better the training the teachers have had, the less likelihood that they'll need a third scorer to resolve discrepancies in their scores. And then some people have asked how soon the tasks need to be scored because they -- in some places they may need to move into the next week. It's my understanding that score reports are not due until mid-July, if that becomes necessary. >> Rochelle: When you say the task-specific, because what teachers have been working with so far are all sample things and we're talking about what exactly is on that exam being given that day? >> Jackie: That's right. >> Rochelle: Okay. You mentioned the informational booklet. What are we going to find in that? >> Jackie: That's right. We have a graphic, I believe, that displays what's in that information booklet. It looks something like the booklet that's available now for the Grade 5 test, if people are familiar with that. There will be one of those going to each principal. There will be information that has appeared elsewhere on administering the test as well as ways of organizing the paper and recording the scores. In addition, there will be information on the steps that should be followed in that task-specific training, and the procedures for scoring exactly, most of which are known by now. And that reflects the need for a scoring coordinator in the school who can manage that paper and provide the training. You'll also find in this booklet the procedures for resolving discrepancies. As most people know by now, the tasks need to be double-scored, so two teachers will each score each paper blindly. Those scores will be recorded and those two scores will be averaged. They need to be contiguous or the same, and if they're not, if there's more than one score point difference between them, then a third scorer would look at the paper and give it another score. If that third scorer agrees with the first or the second, then it's the score that was duplicated that would be averaged. If the scorer doesn't -- if it's another score, then you take the midpoint, the score that's in the middle of the two end scores. So those procedures will be spelled out in the document. You'll also see the scoring rubrics, again. They are also in the facilitator's guide that principals have received. And those are the same scoring rubrics. And also included in the information booklet are a master rating sheet that can be used to record the session that's being scored, the part that's being scored, the name of the rater and the scorers. Then there's also a master class record sheet, and the scoring coordinator would keep track of recording the scores and look at those to see which ones in fact do need to be resolved because there's more than one score-point difference. >> Rochelle: Basic bookkeeping information? >> Jackie: That's right. >> Rochelle: What about what's available on testing day? >> Jackie: On testing day, people will be receiving for that session -- we have a list of materials that people will receive for session one, Friday, the 18th of June. There will be a teacher dictation copy of the listening passage, very similar to the one they receive now for the Regents exam. Then there will be an examination booklet, and that contains the tasks and the texts for each student. On the back of that examination booklet will be a tear-off sheet, again very similar to the one that comes with the English Regents exam now. That tear-off sheet provides a place for students to record their answers to the multiple-choice questions for that session. There's also, in the tear-off sheet for session one, also a place to record the multiple-choice score and the essay scores from both sessions, so that's the place to record everything once the whole test has been administered. So all of that will be on the answer sheet for session one. We won't be providing scannable answer sheets this year. We may do a survey and find out if that's really necessary for 26 multiple-choice questions another year, but we know that schools that use different equipment for scanning them, that may not be possible. But at any rate, if schools want to design their own, they may do so, but they need to have a different sheet for each session. Session one includes 16 multiple-choice items, and session two has 10 multiple-choice items. Those are numbered -- those are numbered 1 through 16 and then 1 through 10, and then each answer has four options that are also numbered. That all sounds rather dry and boring, but I know if there are people out there figuring out how to use their existing scannable sheets, that could be useful information. And that information is also in the facilitator's guide, if you didn't get it. And then in addition to the exam booklet, the teacher dictation copy for the listening passage, the students will receive an essay booklet. This is not a carbonized response sheet like the essay form for the R.C.T., but it's more like the booklet that students have written in for the field tests and the pretests. It's just a book of blank, lined pages. There is a page for notes that students can use in listening to the listening passage, but we will ask schools to provide scrap paper for students to draft their responses. >> Rochelle: No blue books? I remember the old blue books. >> Jackie: These are white with lined pages. And session two is just the exam booklet with a similar answer sheet for 10 multiple-choice questions and a student essay booklet, similar to the one they got for session one. >> Rochelle: Okay. And I'm sure everybody is going to be interested in what kinds of things they're going to get as part of the scoring materials packet. >> Jackie: For the scoring materials, and I think we have a slide to that effect, as well. Yes, we're hard at work, working with teachers in recent weeks, preparing annotations for the anchor papers. You'll be receiving multiple examples of anchors at each level. For score levels 1 and 6, the end points, you'll see two student examples at each level. And for score levels 2 through 5, there will be three sample pieces of student work to represent the range, a range of work at each score level. In addition to that, there will be pre-scored practice papers. So after your facilitator or scoring coordinator takes you through the anchor papers with reference to the annotations, then there's a chance to try that briefly on five papers that have been prescored. There will also be -- the scoring rubrics are the same scoring rubrics, but they'll be included in these materials as well since every teacher will need these materials, whereas that information booklet will only go to the principals. So we'll put those in here again. There will also be an answer key to the multiple-choice questions. Can't forget that. And then the thing that everybody is waiting for, of course, is the conversion matrix. This is the grid that will allow teachers to take the total score on multiple choice items and the total averaged scores on the -- we'll call them essays, even though they're not always essays, the constructed response. You'll take those two totals and look at a matrix that treats each total separately. So it's sort of like looking at the mileage grid on a highway map where you look at one axis for the multiple-choice responses and you'll look at another axis for the total essays on the whole test, and then where the two intersect tells you the student's score on a 100-point continuum. >> Rochelle: Okay. >> Jackie: So there are multiple ways -- not an infinite variety, but there's more than one way of getting any one individual score. So if a student is particularly good at multiple-choice questions, then their scores on the essays might be a little bit lower and vice versa, in order to end up on the same place on the matrix. So that's the way that will be done. And of course the passing score for a Regents exam -- for a Regents diploma is 65, and exactly what -- how a 65 is represented will be decided at the standard-setting -- recommendations will be made at the standard-setting meeting that will be held next week. At that point, a panel of teachers will come together and will look at the same things that most people have seen. They will look at the standards and they will look at the scoring rubrics and will look at student work represented at each of those levels, and they will also be looking at field test results from two forms. So they'll have specific examples of tasks that represent the standards, and they'll be taken through a process where they think about all of those materials and then make recommendations for what kind and how much of that work can be equated with a 65. And then the other score levels or the other numbers on the rubric will be related to -- not on the rubric. The other numbers in the matrix will be equated with certain raw scores on the multiple choice and the essay responses. >> Rochelle: But the bottom line is there won't be any surprises? >> Jackie: There really shouldn't be any surprises, and there's more than one way of getting any one score. >> Rochelle: Okay. Very good. We would like to remind you, if you have any questions, please feel free to give us a call or fax them in. Our numbers are right up there on the screen. We know that many school districts have held training sessions for those teachers who will be scoring student papers; other districts are just getting their training underway. Practice scoring sessions include an outline of the rubrics and a review of the required task and then scoring actual student work against those rubrics. We asked two teachers -- Carol Reynolds, from the Burnt Hills-Ballston Lake School District, and Lillian Turner, co-director of English Language Arts at Niskayuna School District -- to conduct a mini-scoring training session on videotape for us. In this clip, Carol and Lillian review the rubrics for Task 4, reading and writing for critical analysis and evaluation, and then they score a student's work. >> Well, once students have written that essay, it's going to be evaluated against the rubric. Once again, I think teachers have probably seen this rubric. Here it is for them. We wanted to show the whole rubric. Even though we know you can't read it, we know you have this at home, and we are going to zoom in on the parts that are relevant. But we wanted to point out, as a reminder, that this rubric and the rubrics for all the parts -- one, two, three and four -- are essentially the same. That is they all are six-point rubrics, and they all deal with the same five characteristics or dimensions or criteria, however you want to put that. There's always going to be meaning, development, organization, language use and conventions. And the fact that the rubric is the same for all of those parts is really a very, very good thing because it says to students, "These are the important qualities of all writing," not just for one part of the test or just for the test. And I think teachers are beginning to use these rubrics daily in their classroom. >> Right. Which is to the advantage of students and to teachers in preparing their students. >> Right. >> I think one of the things I remember from the very first part of training is to keep in mind that the entire rubric is a spectrum, and there's a line between each of the numbers -- >> Right. >> -- but there's not any solid brick wall. So there are high 4s and low 6s, and the entire thing is a spectrum and the responses within each number come in a spectrum as well. >> We thought we would look briefly at a paper that is a very solid paper, a very good paper. In fact, I think we would certainly score it at the 6 level for many of these criteria. >> Right. >> Shall we read the descriptors? >> Sure. >> Okay. So for meaning, "meaning always means the extent to which the response exhibits sound understanding, interpretation and analysis of the task and texts, and a high-scoring paper will provide an interpretation of the critical lens that is faithful to the complexity of the statement and clearly establishes the criteria for analysis and also uses the criteria to make an insightful analysis of the chosen texts." >> So then development, "the extent to which ideas are elaborated using specific and relevant evidence from the text"-- this is the support, where the student puts the things together. A 6 paper "develops ideas clearly and fully, making effective use of a wide range of relevant and specific evidence and appropriate literary elements from both texts." The descriptors at each level, 1 through 6, change only in the qualities of meaning and development among the four tasks. The organization, language use and conventions that are upcoming are -- not only is the description of the quality the same but the description of a 6 response, 5 response, et cetera, is exactly the same. >> Regardless of what -- >> Regardless of what the task. >> Organization is the third of these criteria: "The extent to which the response exhibits direction, shape and coherence." We probably want to have some discussion about the meanings of those terms in our workshop -- >> Right. >> -- for clarification. The 6 paper again "maintains the focus established by the critical lens and exhibits a logical and coherent structure through skillful use of appropriate devices and transitions." >> Which is an interesting place to focus on kind of large and internal organization in terms of dealing with students in the classroom. This is where the fluency of the piece comes into play. >> Right. >> And language use, "the extent to which the response reveals an awareness of audience and purpose through the effective use of words, sentence structure and sentence variety." The 6 paper is the sophisticated writer, essentially. "Stylistically sophisticated language use that is precise and engaging, a notable sense of voice and awareness of audience and purpose and the student paper varies the structure and the length of sentences to enhance the meaning." >> And it's going to be very, very clear very early on in the paper as to whether or not it has these qualities, certainly. And then conventions finally... "The extent to which the response exhibits conventional spelling, punctuation, paragraphing, capitalization, grammar and usage. The 6 papers demonstrate control of the conventions with essentially no errors, even with sophisticated language." I think people appreciate seeing that conventions line as part of the rubric. It does matter that students know these things and that shows up very clearly on the rubric. >> We know that teachers are going to challenge some of the scoring and so what we're looking for is consensus and agreement overall in terms of what the paper looks like. This one has -- it's been pretty regularly scored as a 6. >> Yeah, and it's important to look at each of those parts, too, to do that analytical scoring, even though eventually we're going to step back and come up with an holistic score ultimately, but we need to look closely at the parts to form that ultimate determination of quality. Do you want to read it, Carol? >> Well, if we just did the beginning paragraph: "In these politically correct times, it often seems that each individual's value system differs vastly from that of the man next to him. However, no matter how diverse two cultures, dogmas or individuals might be, there are some values, some truths that hold universal meaning for all. Great literature, according to Faulkner, calls attention to these all-encompassing, all-too-human themes of 'love, honor, pride, compassion and sacrifice.' I couldn't agree more. Literature which, regardless of its topic, strikes an emotional chord within all of us is the best. Great literature is literature that makes us think, makes us remember, makes us feel." I think in that introductory paragraph, number one, we're clued in pretty quickly to the sophistication of the language use, but also in the latter part of that paragraph, the student does interpret the quote, "literature which strikes an emotional chord is the best," and in doing that he establishes his criteria: "Great literature makes us think, remember and feel." So then he's going to go on to look at two pieces of literature and see how they meet the criteria. >> Shall we continue? >> Sure. "It takes a truly well-written novel to make us understand the emotions of an entirely alien situation. One such novel is James Dickie's "Deliverance." Dickie calls on universal human emotions and experiences of fear, loss and determination to help us to understand an experience unfamiliar to most of us: Pure survival. With his adrenalized prose, Dickie forces us to remember a time in our past when fear gripped us. With these shared emotions as his thread, Dickie weaves an emotional tapestry so rich that even the most extreme circumstances seem not only bearable, believable, but somehow familiar. Most of us have never been chased down a river by a homicidal madman. However, because most of us have at one time or another been afraid or have had to face some emotional or physical struggle, we can relate to much of what Dickie's characters are going through. Thanks to what Faulkner called "universal truths," we too are able to careen down the river afloat on a raft of shared emotions." See what I mean about the language! >> Yeah. This writer has control over sentence structures. There's wonderful variety, wonderful rhythm to his prose, variety in length and structure. And I think he's making it very clear that the stylistic element or the literary element that lets this book or makes this book work in terms of the criteria he's established is the language that Dickie uses, and he really captures that idea here. >> And the student discusses it metaphorically -- >> Absolutely. >> -- with his tapestry metaphor. So there's an example of a beginning of a really outstanding paper. He goes on to do just as well in the rest of it. So using the rubric, you can go across the rubric and point to the things that he does well. He's established the connections; he's developed those connections. Certainly organization and language use and conventions are also in that same realm. >> We need to give that rubric to give credit to what the student does, to look at it as a kind of credit model as opposed to debit model, what he doesn't do. >> Rochelle: Joining our conversation now is Carol Reynolds from the Burnt Hills-Ballston Lake School District. Carol, thanks for coming back. By the magic of television, she's here; she's there; she's everywhere! That was a nice little bird's-eye view of what a scoring session is like, looking at the rubrics, looking at a student paper, and I know that teachers are going to be concerned about scoring accurately, certainly differentiating responsibly between a high 2 and low 3, which I think has been a topic of conversation before. What advice would you give to teachers as they look at papers in that kind of range and really making that fine distinction? >> Carol: Well, of course, that has a very simple answer! (Laughing) I think probably my advice in terms of dealing with any of it is to go back and just deal with the whole process. I think teachers need to be trained very carefully in using the rubric and using it over and over again. Many of us have been doing that for quite awhile. And doing just what you saw Lillian and me doing on the screen before, in terms of saying, "Well, here's what is happening in this place." There are going to be times when it's very difficult to determine because the rubric is, as I said in that clip, the rubric is a spectrum, and within each number is a spectrum. And so while you're looking at it analytically, I have always said that as you look at the rubric and go across analytically, you are going across. Then when you have to come up with a holistic score, you can begin reading down the rubric and saying "basically, this seems to fall more here than there." But it is a difficult call, and that's why you need people working together to come up with a score. >> Rochelle: In the training sessions that you have been holding in your school district, how are teachers dealing with that differentiation in some of the sample papers they have had? >> Carol: Teachers are being very serious about attempting to reach agreement on scores and to really see for themselves. I think that's the hardest thing, to get used to the process of looking at the rubric and using it as what Lillian called a credit model: "This is what a student can do well and this is what we get." So the discussion opens up and it's fascinating, at least to me, to listen to people saying, "Yes, but look at this. Look at what the student is doing in this sentence..." And to me that's why the process and the training is so important. I have had teachers in my own department say to me -- after the last time we had an afternoon training session, one of the men in the department said to me, "You know, this was not something I was looking forward to. This was not something that I even felt we could get comfortable with, but given what we have done, the time we have spent, I am beginning to feel comfortable with the process and I'm certainly feeling that I'm learning a tremendous amount." From my perspective, that was wonderful. >> Rochelle: Very interesting. That's great, a good creditation of what's been going on and how teachers are accepting. Will those kinds of discussions such as we just saw you having with Lillian, will that take place on scoring day or not? Because teachers are going to be working on a paper individually... >> Carol: I think a discussion like what you just saw will take place in advance of that individual scoring. I think that's the kind of discussion that we want to be having in the training. I have said every time we have sat down to train, "We are going to yet again read the descriptor for each of the qualities. We are going to look at the descriptors for each of the scores within, and we're going to think about those." And then when you sit with the exemplars, "Okay, someone said this was a 6. Let's look again at the rubric -- not someone, a number of people said this was a 6, and here is their explanation of that. Let's look at the rubric and make sure. Let's have that discussion now about how we can see that happening, in places where we potentially see it as a low 6, for example, rather than a high 6, but we come to agreement." I see that as part of the training session and perhaps later, when we need a third person, if we do have to come to a conversation about the consensus or the averaging, rather than just doing it -- Jackie described it as three discrete scores, but I can see conversations, especially the first time through. >> Rochelle: That paper, for example, initially someone could take a look at it and say, "Gee, listen to the language. It's just beautifully done. It's a 6," and move on. And yet another teacher could look at it from a very different perspective and say, "The language is really good, but I'm going to say this is all style and not much substance, that I really don't see them answering the question" and, you know, a big differentiation. What do you -- >> Carol: That actually happened in my department with people who -- you begin to listen to the metaphorical language of the high school student in the paper, and I have forgotten exactly what phrase it is but there's one in that paper that just zings an English teacher who steps back and says, "Wait a minute. What have we got here?" Either we love it or "Is this kid for real?" So that led to the interesting discussion. The language use is obviously sophisticated. So then you have to step back and when you have those knee-jerk reactions, that's your opportunity to go back to the rubric and say, "Okay. What are we talking about? If you feel there's no substance there, what is it we are talking about?" In our department there was significant agreement that it is probably a low 6 or high 5 for development, that particular paper, and yet that doesn't take away from it being overall a 6. As a matter of fact, when I trained within the department in another setting I considered not using that one and using one that was easier to see as a 6... nowhere as good with the language use but in that range. But I think given the fact that most of our students are not in fact going to be doing 6s all the way down, that this was a good choice. >> Rochelle: But it's good to have that difference of opinion and "let's sit down and explore why it's one or the other." You have been doing a lot of training in your school district. Jackie, what about teachers out there who are concerned that they have not received any training or that they haven't received enough training that they feel comfortable yet? >> Jackie: Well, if they really have not heard from their BOCES or missed it for whatever reason and haven't heard anything about training sessions, then the first step would be to check with their BOCES and see if they missed something or, alternatively, to find out if there are trainers available who might be willing to set up either another session or is having another one that they could come to. Also, we have got probably close to 500 teachers in the state who have been involved in the pretest and field test scoring, some of whom may feel comfortable and even eager to provide assistance to districts who somehow missed out. And then if all else fails, if there aren't sessions established that teachers can go to, we do have a few copies of the training materials, if this is something that a small district feels they need to take on by themselves... then those materials, using anchor papers and such from the sampler that are being used in the statewide training, we could send one copy of those to districts who needed to do something on their own. >> Rochelle: Okay. Way back a year ago, last July I think it was, I attended with some camera people a training session where teachers were sitting at tables, looking at rubrics, looking at student work, a lot of discussion and a lot of debate. At that time there was still a lot of discussion about the rubrics themselves. "We don't like this rubric. We think this should be changed..." Have we gotten beyond that? This is the rubric; this is not the time to discuss the rubric. >> Jackie: I really think we have. That rubric has been in development since 1995, and every time teachers came together to score anything, whether it was the pilots or the second year of pilots or the pre-test or the field test or whatever, they have had an opportunity to suggest refinements. And that's what they have been. We're still working with the same qualities, defined in the same way. We have changed the number of levels and we've refined the language in response to teachers' concerns or questions or requests for clarification. But there haven't been enormous changes made all along, but the last use of that rubric, teachers were saying things like, "Please don't change this again." (Laughter) So there comes a point where -- anything that's written can always be revised, but there comes a point where you need to say, "This is it. It's serving its purpose; it's serving it well." The rubric doesn't stand alone anyway. The rubric needs to be accompanied by trained users and by anchor papers that demonstrate those criteria. So taken together, we really have found that the scoring can be quite reliable and that's the reason we ask for the two scorers as well, so that teachers don't need to feel, "Gosh, this is just me and I'm not sure I have understood this completely." There is a lot of support in this process that should make for some more confidence on scoring day. >> Rochelle: What about support on scoring day? There's always the concern that when we have something new or revised or some changes or whatever, that that day something will pop up that hadn't been thought of before or at least hadn't been addressed in this area. On scoring day, is there an 800-number or someplace at S.E.D. that somebody is going to be able to call and say, "Oh, my God, what do we do here?" >> Jackie: You mean I shouldn't plan to go on vacation that day? (Laughing) We won't have an 800 number that I know of, but our phone lines will be open. >> Rochelle: And operators will be standing by, right? (Laughing) Okay. We have said it before; we're going to say it again: This Regents test is directly related to the learning standards which forms the basis for school curriculum. So what's being taught in classrooms is very important. During a teleconference in March, we discussed what some teachers are doing in the classroom to prepare students, and we'll review some of those suggestions for you now. I'm going to go back to Carol and Jackie and ask them some of the things that we have seen going on in classrooms. What's going on in classrooms at your school district, Carol, where teachers are really working to get kids ready? >> Carol: Probably the most obvious thing that you'd see, because our district has been involved with the state since this began, that we have asked all of the teachers, Grades 9 through 12 in the high school, to make sure that each semester every student encounters a sample of each of the four tasks. So every student will encounter four tasks in the fall and four tasks in the spring. So students have that background. But from my perspective, that's not the most important preparation at all. What we have been doing, and we had summertime to design those tasks to fit with the curriculum, but I think now what we're doing is we're moving beyond that. It's not the task focus so much as the standards focus. So we're stepping back and we're looking at the standards and we're building those -- just making sure -- they have always been part, I think, of our curriculum and the things that teachers are doing in the classroom, but making sure that they're there and that consciously we're putting those activities in there. So as I have mentioned before, things that might be different because of the new assessment... teachers put quotes on the board or journal entries very often will be a quote that's about literature, where students respond to that quote in their journals and just talk about what it means. We have some students who are not used to that and that's going to work against them on task four if they are not used to taking someone else's work and putting it in their own words and talking about whether or not they agree or disagree. I think we're doing more with graphics and analyzing graphics. In one of my courses where in the past I would give an article that was about immigration and changing demographics, I would give them the article and the graphic together. Now what I do is I give them the graphic and I say, "I want you to put down everything this tells you; jot that down first," and then we read the text, just to get the focus on something like that. >> Rochelle: The idea, that you're saying an article, what we're talking about then is more "real life" reading? >> Carol: Right. The kind of reading for information and understanding that the first two parts of the exam essentially address, although it's listening in the first part. Surprisingly, many English teachers, even in my own department, had worried about that part, those parts of the test. When we have used tasks, we began to realize the degree to which that's the kind of activity students are doing in other classes, and they are transferring that in. So I think we -- I know I personally have been pleasantly surprised at the degree to which students can be successful on that part, even some of the students we were concerned about in terms of the test. We have made conscious efforts to get the rubric into the hands of students. We -- you know, we want the students to know what are the qualities on which they're going to be assessed? How are those defined, and what does it look like when it's good and what does it look like when it's not done as well? One of our teachers will have a student write an essay, take the rubric, and you have to get in a small group and you have to have two or three other students read your essay and do the analytical scoring of your essay. That student has written the same essay. Now, from my perspective, you know, once you step back and have to analyze somebody else's, you really have to look at the essay and you have to look at the rubric, and in the back of your head you're thinking about what you did. And so then they sit down together in a group and talk about the scoring, and I just see that as a wonderful learning experience. >> Rochelle: What were students' first reactions to the rubrics when they saw them? >> Carol: "What's this?" "Oh, no!" Although, that's not entirely the case because this is just one of many rubrics that have been around. They're used to getting rubrics with a number of their assignments, and not only in English. I think that's been a wonderful step in education, that we have made kids aware of the criteria for evaluation so they have those in mind while they're working. Just today Jackie gave me another idea that had not even occurred to me, which is to begin asking the students to use the five qualities in analyzing works that they encounter, so if they're reading a short story, instead of talking about -- however the discussion might normally go, to use that, you know, "What's the meaning? How is it developed? How is it organized? What can you say about the language use and the grammar and the literary elements?" And that's just another way of embedding that, and that's what we have tried to do. >> Rochelle: The literary elements, always part of the discussion as well. What are those literary elements? And having students work with them on a regular basis, practice with them so that they come quickly and easily to mind. >> Carol: Probably much more consciously we're doing that than we might have been doing that until recently. >> Rochelle: Jackie, what other types of things do you see teachers do or recommend that teachers do or don't do? >> Jackie: I think the critical thing that Carol already alluded to, and that's to think beneath the surface of the exam and what it looks like to the real question of what kinds of language behaviors does this call for? Why these tasks? And to think about the standards that lay behind the exam. For example, the Part 4, the critical lens, it isn't just about using a lot of quotations, although that certainly is a useful way to get kids to think about how they evaluate, how they form judgments, but that whole notion of recognizing that as our criteria changes, the evaluation changes. So to be thinking about their assumptions and thinking about their criteria, using and coming up with their own criteria for making -- drawing conclusions and forming opinions and seeing what happens if you look at a text from one set of criteria versus another or evaluating a movie from one set of criteria versus another. That's what that task is getting at, the use of criteria to form judgments and opinions. And Part 2 isn't just about adding graphs and charts to the English class; it's about being able to synthesize information from more than one source. We expect kids to do that every time we send them to the library or do research. So this is a proxy for those behaviors, knowing what information to select, what to do with it in terms of organizing it, which of it is relevant to this task, transforming it for another audience and another purpose. So those are the kinds of things that need to be at the forefront of any other kinds of assignments, whether they are tasks that look like these or not. >> Rochelle: The danger is that teachers will just clone things from the sample test... >> Jackie: That's right. The intent is not to create an English curriculum out of these four tasks. >> Rochelle: My recommendation would also be -- from my non-education background but from my student test-taking background, would be have the kids list books or pieces of literature that they have read through the year because the big thing is you get into that test, and it says "draw on literature you have read during the year" and all of a sudden your mind goes blank and you think "Oh, my God, what did I read?" And your inclination is to draw on the last couple of things that you read as compared to what you picked up last September. >> Jackie: And their responses will be richer if they do draw on material that they have studied rather than something that they read on summer vacation four years ago, you know. >> Rochelle: All rightie. Another support system that teachers now have is the booklet that was developed to accompany this program. This is the booklet that we're talking about, "Testing 1-2-3," a nice booklet that has a lot of helpful information in it. Jackie, what are some of the things that teachers should make sure they make a point of looking at in here? >> Jackie: Well, certainly the purpose of the test. >> Rochelle: (Chuckling) Why are we here? >> Jackie: Right. Why these particular tasks and how they're related to the standards. So looking at the map to the standards, I think, makes it quite clear why those particular tasks are used and how anything less would not be a good representation of the standards. I think teachers will find the glossary of terms that are used on the test to be helpful. How do you define a critical lens and what is a controlling idea? Those are not foreign terms, but they were used for particular reasons and seeing the glossary might be helpful. The chart on Page 19 may be a succinct way of thinking about either a district curriculum or one's own course curriculum, thinking about the texts that are required that address each purpose for language and the strategies that are intentionally taught that address each purpose. >> Rochelle: That page is "A Review of Learning Standards in my Classroom." >> Jackie: Yes, and the chart opposite that. So that if you're planning a unit or a whole curriculum, you're always thinking: What uses of language are we engaging our students with? What products are we expecting of our students that fit into each of those purposes? What assessments am I requiring that address those purposes? And so forth. No one could write a curriculum in a page that size, but there's a larger version that could in fact make up a curriculum that's based on the standards. And then the information on scoring on Page 25 kind of summarizes some of those things I shared a little while ago about the test materials and the scoring materials. >> Rochelle: Some of the things that schools can be doing now: Appoint a scoring coordinator; provide task-specific training; assign two teachers to score each task with a third teacher available to resolve discrepant scores... that sort of thing. >> Jackie: Right. >> Rochelle: Very good. Any other last thoughts that we may want to pass along to teachers as they're getting into the winding down of the... >> Jackie: Well, I know that one question that people are certainly thinking about is what to do with -- how to help the students who may not be successful this first time. And, of course, they will have other opportunities to take this test. It will still be offered in August and January and next June, next August and so forth. But at the same time, there is a requirement in the new part 100s calling for help to be provided to students who weren't successful and, of course, teachers are doing this all along anyway. When a student fails a test, then they'll provide the help they need. But I know that Carol and her staff have been thinking about the kinds of things they might do to set up a program for those kids. >> Carol: Right. We have been trying to decide what's going to be meaningful for remediation, knowing full well that sitting down for a semester and doing task after task after task is not in fact meaningful or beneficial to anyone and would probably only increase anger all around. So what we have been trying to do -- the reality of the school world, we don't know how remediation is going to be scheduled and what we'll have for staffing in order to provide that. And so what we're trying to do is put together a program that would have a student prepare a portfolio that would show his or her performance in this, and we're designing it to be -- to make it show that they can meet the standards. As I have begun trying to map this out, I have a chart something like yours there, a chart saying that there are certain activities that the student is going to do to meet each of the different standards, and because the remediation is outside of the normal classroom, the topic isn't mandated. So if it's a ninth grader who's had difficulty with the eighth grade exam, it doesn't have to do with mythology, which might be the unit he's studying in class. In my idealistic little world, I'm envisioning a portfolio where the student can pick something that he or she is interested in, a topic, probably more than one, and do exploration within that area. So if you're fascinated by -- I talked to one of my students who might have trouble with the exam, and I said, "If you had to do this, what would you do?" "Oh, I would do local bands." "Okay. Now, how would you do that, Jim? Chat with me about this." "Oh, there's a lot of stuff. I know where they play; I have seen them play; I have their C.D.s; I have a lot of information about it; I know the magazines..." And I think this is something this young man could in fact do and probably would do far more willingly than reading "Lord of the Flies." Those are the kinds of things, that you can design activities that he can do that are focused on his interests. So it's meeting the standards and I think any of the remediation is going to have to involve some concrete things such as examining a variety of graphics a student might encounter, some real focus on literary terms and finding them and doing it as lesson sorts of things, and also it has to include that the child has performed the different tasks and is familiar, very familiar with the nature of the tasks and can be successful. So we're looking at something like that as a way of building a program so that it is something that is outside of the normal classroom but appeals to the student's interests and reinforces the kinds of activities that are specifically going to lead to this kind of underpinning for them as they go into the test. >> Rochelle: But the bottom line is that school districts need to begin to think about this and really begin to make some plans for it. Okay. The other question that often comes up, and we'll run through it again quickly, is special education students or students who require accommodations with the exams. Do you want to run by those requirements? >> Jackie: Well, as usual, students who have I.E.P.s will be allowed to have those accommodations and modifications that are indicated in their I.E.P.s. They will not be able to eliminate things like the listening portion, for example, for hard-of-hearing students. The issue there is how do those students understand or demonstrate their understanding of material that's presented in sign or orally throughout their school years? And so the accommodations need to take that into account. For example, one accommodation might be to hear the listening passage or rather have the listening passage signed more than once. Another might be to stop at key points, after each paragraph if necessary, and either stop to take notes or stop to ask a scribe to write down some key words. There's no requirement to take notes, so if notetaking is not part of the hard-of-hearing student's repertoire of understanding non-print material, then you wouldn't want to require it for this test, but you do need to find other ways for them to have the time and the wherewithal to process information in that form. It's not appropriate for that hard-of-hearing student to, as I said, have that part eliminated. It's not appropriate to give them a copy of the listening passage because then it becomes a reading comprehension passage, and it's not appropriate to have notes given to him from somebody else because then it's somebody else's comprehension assessment rather than that student's. Students who have disabilities in spelling, for example, by the time they're juniors in high school, that would have been reported and accommodations made for that in other assessment situations. For example, those students might have one of those Franklin spellers or they might be able to use a word processor with a spell-check in it, in which case their papers would be scored just the same way everybody else's have been because that accommodation has already been accounted for, has already been made. Those are the ones that I usually get asked questions about. But it's whatever the C.S.E. has determined is appropriate for that student, is used consistently in all their assessments and is recorded in the I.E.P. >> Rochelle: All right. We want to make sure that everybody -- if you would like some additional copies of "Testing 1-2-3," we do have some available. You can get another copy of the "Regents Comprehensive Examination in English." This is the facilitator's guide. Please contact Jan Christman. Jan can be reached at (518) 474-5922. That's Jan Christman at (518) 474-5922, and that is for a copy of the "Regents Comprehensive Examination in English Facilitator's Guide," which Jackie has noted has some good information for you that you'll probably want to continue to work with for a little bit. That is all the time we have for this evening's discussion. If you have any questions about the English Regents exam, scoring or training for scoring, direct them to the following E-mail addresses or fax numbers and we'll pass them along. E-mail us at either tmoore@questar.org or jquinn@mail.nysed.gov . We'll pass your questions along to Jackie. The fax number is (518) 477-4311. About some programs coming up in our schedule, I want to let you know that tomorrow night, State Education Commissioner Rick Mills will talk with parents about school report cards in our "New York Learns, Parent Night" series. The program airs from 6 to 7 p.m. and can be seen on public television stations across the state. Among those joining the Commissioner and me are the head of the New York State Business Council, a print journalist, and a superintendent of the Rochester City School District. We're going to explain for you how parents can interpret the information contained in the school report card. We're going to talk about why report cards are an important part of helping to improve student achievement because the bottom line is that's what the bottom line is: Making sure those report cards translate into better student performance. On April 28, "New York Learns: Test Review" returns with a look at the new Math A Regents. Lynn Richbart and Jackie Marcano from the State Education Department return for a look at this exam that may be optional this year but is the exam that students will be taking in the year 2000. That program airs from 8 to 9 p.m. on the station you're watching now and at satellite downlink sites. It also will be live, so we'll be looking to you, our viewers, to telephone or fax in your questions and your comments about the exam. On May 5th, our "New York Learns" series wraps up the school year with a teleconference on technology in the classroom. Our discussion is going to go beyond putting computers in labs or in the back of a classroom. We want to show you how teachers can incorporate technology of all kinds into classroom instruction in a way that's designed to help raise student performance. That program airs from 8:30 to 10 p.m. on your satellite downlink sites and many local cable access stations. And in response to requests by teachers who would like to receive training in scoring the 4th and 8th grade Math or English assessments directly from Measurement Incorporated, we've made arrangements to provide that training via teleconferences on June 7th and 8th from 8:30 a.m. to 2:30 p.m. "New York Learns" will air training provided by Measurement Inc. representatives. These programs will be live, so you'll be able to call in your questions directly to the people who helped New York State develop these exams. That training will be in the math exams. My thanks again to Jackie Marino and to Carol Reynolds for their work on tonight's program and to all of our viewers. I'm Rochelle Cassella. Enjoy the rest of your evening. (Music) |