Transmitted via satellite, produced by the New York State Education Department and the New York State Satellite Distance Learning Network; broadcast from the NYNET studio in Albany, New York.

For additional information, call or write:

John Quinn
Room 668 EBA
Office of Curriculum and Instruction
New York State Education Department
Albany, NY 12234
(518) 474-3954
E-mail: jquinn@mail.nysed.gov

(Music)

>> Rochelle: Good evening. This is a "New York Learns News Brief." I'm Rochelle Cassella.

The tests have been given, the scoring done. All that's left is to wait for the results of the fourth grade English language arts assessment. After months of preparation, speculation and concern, students across New York State took the exams in mid-January. Bad weather, both in sections of New York and the midwest, where the test came from, played havoc with the schedule for giving exams in some schools, but by scoring days last week, the tests had all been given.

Districts also had concerns about the logistics of regional scoring, and among those concerns, the need for substitutes in classrooms so that teachers could score the papers, the need to train scorers on scoring day, having enough scorers available, and the logistics of moving boxes of test papers from schools to scoring sites. For the most part, however, scoring day went smoothly. Here's what it looked like at one Albany-area site.

>> The purpose is to assess how we're doing on our new State standards and if students are doing well, they'll do well on this test. If they're not doing well -- if they're at a two or a three, then the rubric says this is what you need to do to go to the next level.

>> Steve Abelson is in charge of training teachers on just how to score the new standardized test for fourth graders. More than 150 third, fourth and fifth grade teachers are here in Albany. They represent 23 public and private school districts, in Rensselaer, Columbia and Green county. Since last August, they have undergone extensive training. This half-day training session is their last before they actually begin scoring tests.

>> I am the terrible slayer of lions! Hurry, lion, eat the hunter so I may eat you.

>> The teachers are broken up into three groups, or domains: one for reading, independent writing and listening, the areas the students were tested on. There are about 45 teachers in each domain. Each group is led by a scoring leader with site assistants and table facilitators to help hand out papers, test booklets and to answer questions. Because the tests are based on certain task-specific rubrics, scores are based on how well students show their work as opposed to giving multiple-choice answers.

>> In old standardized tests, students get a 91 or maybe an 89. What's the difference, other than two points, between the two scores, and what do those two points mean? In a rubric score, you can go from a zero, which means you didn't write anything, to perhaps a 3, which means that is exactly what we were looking for.

>> In each group, scoring leaders walk teachers through the process of what to look for using sample papers. They also rely on anchor papers, pilot test responses they can refer to when scoring.

>> I didn't know what to expect. I was nervous. You know, you want to do the best by the children. But sitting in on it, a lot of my fears have been alleviated so far.

>> It's not difficult. I think the problem that most teachers have is the subjectivity of it. Whereas one person might rank it a 2, another would give it a 3.

>> We've got difficulties now with the groups in the room, but we have the entire morning to work those difficulties out, and so we're going through the process now of all getting into the same frame of mind as far as scoring all the questions.

>> As part of quality control measures, some student responses will be chosen at random and given a second read by a different scorer as a way to check for accuracy and consistency. Scorers also place a personal code on all papers. What began as apprehension turns quickly into confidence as the training continues on and eventually comes to an end.

It's now time for the actual scoring to begin. Site assistants, volunteers from nearby St. Rose College, pass out boxes filled with test booklets. Each box contains 60 tests. Teachers are also instructed to look for any student papers that might reveal problems that authorities should be made aware of. And while much discussion has taken place during the training sessions, the actual scoring is done in silence, with the only conversation being between teachers with questions and scoring leaders, site assistants and table facilitators. All together there are more than 2,000 tests that have to be scored. Each teacher will score 50. Now, some of the papers will be seen three times by each domain and some others will be seen six times.

>> The only problems I have seen are people asking questions, but they are more competence questions: "Is this okay?" and "Do I have the right idea?" And as they gain confidence, in the last hour, you have seen less and less of the questions.

>> You saw some similarities in the answer and you know then what you can pinpoint and tell the children they need to fine-tune, the specific details that tie things together. So I think it's the beginning of a good thing.

>> It's quite a change from what we were used to, and I was a little apprehensive about just what to expect and what you would do, and having done it, I'm much more relaxed. I learned that it's possible to look to the future and just feel better about it.

>> I think that it's a good idea. I think having standards is a really good idea. I think having all the children learning the same way and learning to answer, I think the way we're teaching the kids how to answer questions and how to think and make the connection, that will benefit them in the long run.

>> After more than three hours, the job of scoring is done. The tests will be sorted out and placed back into boxes designating their home schools. Final results will be completed by July.

>> Rochelle: And those results, as Wanda indicated, due back later this year, will help teachers determine where their students need additional instructional support.

New York State Education Commissioner Rick Mills provides parents with another look at the revised comprehensive English Regents exams being given this June when he holds another in his series of "New York Learns Parent Night," this on Thursday, February 11th. The program airs on public television stations throughout the state beginning at 6 p.m. It's a live broadcast so parents can call in their questions and comments.

Public hearings on the new State assessments, proposed graduation requirements and annual teacher performance reviews continue across New York State this month. Individuals wishing to comment on these changes may do so on the dates and at the locations you see on your screen. Anyone wishing to speak must register prior to the opening of the session. Comments are limited to three minutes per speaker.

To receive a copy of the proposed revisions to Part 100 of the Commissioner's Regulations, check with your local BOCES office, or you can access the Education Department's web site at http://www.nysed.gov.

Individuals involved in services to children with disabilities are invited to participate in focus groups that are being sponsored by the Federal Office of Special Education, the New York State Education Department, and the New York State Department of Health. Data collected from these focus groups will be used to assess New York State's system for early intervention and special education services. Topics to be covered include early intervention, special education programs for students, and transition from early intervention to preschool special ed.

The public meetings will be held from 4 to 7 p.m. in the following locations on the following dates: Monday, February 8 in New York City and Westchester; Tuesday, February 9 in Long Island and Buffalo; Wednesday, February 10 in Syracuse and Saratoga.

Check the State's web site at http://web.nysed.gov/vesid for the complete details.

The start of the new year also saw the beginning of field testing for career plan prototypes at the 18 sites the Office of Work Force Preparation and Continuing Education selected to participate in the Career Plan Pilot Initiative. Field testing continues through the year with a pilot report to the Board of Regents expected in the spring of the year 2000.

Once the program is evaluated, the Office of Work Force Prep expects to have identified the key elements of the K through 12 Student Center Career Plan. It will recommend those standards for statewide implementation. A career plan helps students document who they are, where they're going and how they'll get there. At three different times during their education, students will be able to summarize their academic growth, their achievements and their future goals.

That's "New York Learns News Brief." I'm Rochelle Cassella. Stay tuned with us for "New York Learns: Teacher to Teacher." We'll answer your questions about Comprehensive District Education Plans and our discussion tonight is on the revised comprehensive English Regents exam being debuted in June.

(Music)

>> Rochelle: Good evening. I'm Rochelle Cassella. Welcome to "New York Learns: Teacher to Teacher." Right now we're talking -- in a few moments, we'll be bringing up our discussion about the comprehensive English Regents exam that's coming up in June. But first we're going to answer some of your questions about last month's topic. That was the Comprehensive District Education Plans.

Back to help answer your questions: Dick Jones, Team Leader for Regional School/Community Services, and Barb Flynn who is supervisor with Innovative Programs from the New York State Education Department. We're delighted that you could both come back this evening.

I do have some questions that were faxed and E-mailed to us between now and last month's program. But I want to remind our viewers that if you have a question about Comprehensive District Education Plans, you can call us and ask it live by calling 1-877-280-7400. That's a toll-free number. Again, that's 877-280-7400.

One of the things that I did get some questions and some calls about, some things that were faxed and E-mailed to us, and certainly one of the things that people wanted to talk about a little bit was the software program that we had mentioned and showed during last month's program, that there is a software package available and there was some training done. The individual said we didn't really get into any details and they wanted a little bit more information.

Dick has told me Barb is going to address this question for us. Barbara, a little bit about the software package. How does it work? Is there cost involved with acquiring the software? Give us a little bit more information.

>> Barbara Flynn: The software package is designed to parallel the planning process that we've created. And what it does is not only leads people through a planning process but gives them many, many more in-depth questions and tries to get them to expand their plan. It is a complicated software program. It was developed in England, actually, first for New York City and then changed to reflect what the state was doing. It is available to pilot districts free for the first year of the pilot, and beginning next year we are going to have to charge for it.

One of the reasons that it's been successful is that we have provided monthly training sessions for everyone using it. It's a complicated program, and unless people are really computer-proficient, they have a lot of trouble with it. I think the thing we have learned is that it works for some districts and not for others, and it depends on how they want to work with their planning committee, how much planning experience they have, what kind of computer support they have, whether they have people that really love to play with computer programs. And so it's a tool.

>> Rochelle: Okay.

>> Barbara Flynn: But it works for some and not for others.

>> Rochelle: One of the questions asked was -- we talked last month about having a facilitator, an outside facilitator to come in to work with your group to help you do the planning process. So the question was would using the software program negate the need for a facilitator? Does it help you get into those in-depth questions the way we talked about last month that a facilitator would?

>> Barbara Flynn: Not at all. You still need a facilitator; you still have to go through all the group dynamics and getting various people to look at all the questions, to bring them together and focus the discussion. The software is simply the place where you can put the answers and compile it, so -- it does provide the questions, but it does not replace the facilitator in any way.

>> Rochelle: So you're saying it would work for some school districts and not for others. How does a school district at the start of this know if this is a good tool for them to have, if it would work for them?

>> Barbara Flynn: Most of them don't. They have to really at least try it. I think about half that try it have used it and half haven't because they found it too time-consuming or too labor-intensive.

>> Dick Jones: One criteria might be whether they have an established committee already. That's a big leap for some of the districts, putting together a structure. But districts who already have a structure, they might be in a better position to tackle it using the software that will take you through a greater in-depth analysis of some of the things that will be a part of your plan.

>> Rochelle: Okay. So a district that already has some structure in place as opposed to a district that's starting from ground zero, haven't even gotten a committee together...

>> Dick Jones: Certainly, starting from ground zero, it becomes just one more hurdle.

>> Rochelle: (Chuckling) Right! Okay. And you were saying that if a district decides to attempt it, there is some training available for it?

>> Barbara Flynn: There is monthly training available throughout the state. We're trying to figure out a way now that we can continue that kind of training and really grow our own trainers in New York so that there will be far more people able to help districts that want to use the software.

>> Rochelle: Do you have any idea of how many districts so far have used the software?

>> Barbara Flynn: Well, in the first round we had ten plans out of thirty-five that used the software. I think we have, in the current pilot of about ninety districts, we have forty that are using the software. So it's becoming more popular.

>> Rochelle: Have you seen any difference in the initial quality of the plans from the districts using the software as opposed to those who haven't?

>> Barbara Flynn: Not really. Good plans are good plans, and the districts that have pushed it farther, whether they were using the software or not...

>> Rochelle: So it would just be an additional tool then.

Once again, I want to remind viewers that we are answering some questions and providing some additional information with regard to last month's program on Comprehensive District Education Plans. If you have a question you would like to ask about that topic: 1-877-280-7400 is the toll-free number. We'll be talking about the comprehensive English Regents exam in our next half hour.

One of the other questions that came up was with regard to expense. Somebody said in our videotape of the orientation session that the state has -- there was some discussion, a little bit, about what it might cost a district to go through this Comprehensive District Education Planning process. The overwhelming response from those people on the panel that you had in your orientation session was that it's well worth the expense. But I did have a couple of districts who wrote, "What kind of expense are we talking about? Is there any average amount that a district would spend? What kinds of things do we need to take into account expensewise when we get into the planning process?"

Dick, any information on that?

>> Dick Jones: Well, again, it varies; it's hard to put a specific amount on it. The real cost is more the time, the staff time to devote to some of the planning and bringing in the committees that are necessary and time for those groups to meet. The real expenses might be, if you have an outside facilitator, paying for their time and any necessary expenses that might be incurred in bringing a group together, if you need to provide meeting space and reimbursement of some of their expenses for those groups to meet.

So if you're looking at the absolute cost, it's hard to determine that. But really the greatest commitment that districts talked about is freeing up the time and devoting to the process.

>> Rochelle: So it isn't like putting together a Comprehensive District Education Plan for a large district, in a large district is going to cost you $10,000 or something? There's no average figure for what it may be?

>> Dick Jones: No, I don't think we have got that feedback from the schools.

>> Barbara Flynn: I think what we've heard is that there are some costs involved. They're the kind of costs the districts would have if they did any kind of planning; for example, they have to hire substitutes so they can free teachers up. We had one district that talked about the fact that -- one superintendent said, "Well, what I had to do was provide baby-sitters for the low-income parents that were on my group, but in fact what I did is I made my teenagers baby-sit for them and then I paid them."

So there are some hidden costs in some of these things if you really want to get a lot of participation. What we basically heard is that it may be expensive the first year in terms of really involving more people. But most districts think there will be cost savings down the road because they do not have to do as many plans; they're able to incorporate almost all the other plans they have in the state in this plan. So while it may be very labor-intensive the first time, it may not be that way in the future. So they really think in the end there will be cost savings, but the up-front costs are pretty big the first go-around, mainly in terms of the time.

>> Rochelle: Is this the kind of thing that you're asking school districts to track over the long run so you can tell if there is indeed a cost savings or if it's a lateral kind of thing, that you don't necessarily lose anything but it doesn't cost you any more for the fact that you have been able to incorporate some of those other plans into the comprehensive plan?

>> Barbara Flynn: Some districts are tracking that, yes.

>> Dick Jones: And as part of the just general feedback, asking them what their experience was? What was good about the process? What could be done differently? That's what we're trying to collect from data. So some of that may not be quantified in exact numbers, but certainly whether they thought it was worthwhile.

>> Rochelle: Okay. Once again, you can call in and ask about Comprehensive District Education Plans, if you would like. The number is toll-free: 1-877-280-7400.

What's the latest information or the latest talk that we hear -- right now this is a pilot program and the whole idea of doing a Comprehensive District Education Plan or C.D.E.P., as some districts have been known to call them, is voluntary, with the idea that at some point it might become mandatory. What's the latest talk on that? Have you heard anything more about...

>> Dick Jones: Well, that's the question we always get: Is this going to become another mandate that we need to pay attention to? It is very much a pilot program. We're trying to get experience from districts about what works about the process and within that, a range of districts from which we're getting some experiences, those who have had planning processes before and those that this is a big hurdle for them to develop a different type of extensive input for developing a plan.

As Barbara said, one of the things very definitely we made a commitment to is that this will replace a number of plans that are already required. So in that case, it's going to be a reduction of some of the efforts of the districts, that they can consolidate some of the things they are already mandated to do. We haven't made a definite decision as to whether this will be mandated.

My sense is that it probably will become a mandate at least for some districts at some point in time, based on the planning requirements. But we still need to collect more experience on how well it plays out and how it works in all types of districts before we make that decision. So far the feedback has been very positive with people talking about the process. So we're confident that it's going to be something that will be an advantage for every district to participate in, whether it's mandated or not.

>> Rochelle: How long do you think it will be? What kinds of information are you going to be looking at from the districts before you make your decision?

>> Dick Jones: Well, I think after we receive the next round of proposals and, again, that gives us additional groups that we've had some experience with and we're continuing to broaden the types of districts that have been involved in the process.

>> Barbara Flynn: I also think there's going to be a lot more desire on the part of districts to do a comprehensive plan. This year they had to do another new plan, the operating standards aid plan, and next year they're going to have to do a mandated professional development plan. And both of these can be folded into a comprehensive plan, so there are a lot of things pushing districts in this direction.

>> Rochelle: Okay. So the idea is as those other plans come, everybody is going to like this one.

Let's talk about the most recent round -- you're getting some plans now. Excuse me. The flu season is going around and those little tickles are getting in there.

We're getting some plans in now?

>> Barbara Flynn: That's right. We're expecting about 40 plans to be filed between now and the middle of March. We'll be reviewing those at the end of March. Those will be plans that will be in effect for the year 1999/2000, and we then we should have about 80 plans coming in by the end of July, which will also be for the year 1999/2000. So we're going to be up to about 260 districts.

>> Rochelle: How many in the state?

>> Barbara Flynn: 740-some.

>> Rochelle: So you're getting them in there. So the review process for those districts who have submitted plans or are submitting them now, they would expect to hear sometime in April, May?

>> Barbara Flynn: Sometime in April.

>> Rochelle: Okay. Once again, let's talk a little bit about that review process. We bring in some people who have been trained on how to read the plans and what to look for. A little bit about their training and what kinds of things they're looking for?

>> Barbara Flynn: Well, what we have done is had people from the field come in and help us review the plan. And they were either people from BOCES who had been involved as facilitators or some other way with comprehensive planning or representing a school district that's actually done some comprehensive planning. We set out a structured review process for training people. We told them what we were looking for in the plan, how to review them.

We had groups of three people spending about three hours on each plan that they reviewed, discussing it, seeing if it met the criteria, where the weaknesses were, where there were improvement opportunities, and where the strengths were. And the process went very well the first time, so I expect we'll repeat it, improving it, of course, but repeat it in March or early April.

>> Rochelle: Have there been some changes in some of the things that you're looking for or are asking for in the first time -- as we talked about, this is a pilot program, so we're growing and we're learning and we're changing, right? So what kinds of changes have we seen in what we've asked for before and what we're looking for this time around?

>> Barbara Flynn: Well, I think the main changes are really going to come after this stage, as we evaluate the group that's coming in right now. At that point, we'll be revising the guidelines for comprehensive planning for next year and really looking to see how we can incorporate some of these new plans like professional development plans, to structure them in the guidelines to make it much more easy for districts to complete it.

>> Rochelle: Great. Great. So then we would expect an orientation session sometime next fall for the next round of plans? How might that work?

>> Barbara Flynn: I would expect there will be another orientation session in September for the districts that want to join the pilot, or really the program next year. I think we're going to have to stop being a pilot at that point!

>> Rochelle: And when we get that information from Barbara, we'll make sure we make it available to school districts through programs such as this one in the news brief.

We're hearing a lot of good things back from the districts. Dick, are you finding that those districts and those administrations who have been through the process are your best advocates for getting the word out and telling other districts to take part in this?

>> Dick Jones: Oh, absolutely. We don't go around promoting the planning process, but people don't necessarily trust us in saying this is something worthwhile. It's really been the words of many of the people that have experienced the process. That's who other people trust, and they have been the best advocates. And we have had a variety of forums where we've had opportunities for administrators and staff and parents and teachers that have been a part of the process to talk and share with other districts. They have become the most positive advocates for us.

>> Rochelle: Okay. Parents must find this fascinating. You learn a lot about the districts and how the school district works.

>> Dick Jones: Well, it is. And that's one of the things that some districts have changed and opened up the process. We have had many parents involved for the last several years in building-level planning, and that becomes a little different perspective. There, you're pretty much focusing in on some specific needs within that building. And within the comprehensive planning process, we're getting more to the aspect of some of the budgeting, how some of the resources are used and, more importantly, we're asking to make decisions on that basis, based on some overall student achievement data from the entire district. And administrators have frequently been a part of that process, making some judgments. One of the things we have encouraged is to really open up that process and have more parents involved. It takes a little time for them to get comfortable with looking at that volume of information, how to work with the data, and that's been a part of the planning process. But those who have been part of it find it very rewarding. They become part of making some of those decisions.

>> Rochelle: Before we leave, I know that one of the other questions that we had come in was a follow-up. Now, once the school district has submitted its plan and has gotten feedback from the state and is actually starting to implement the plan, what will the state do? Is there a regular review process that will go out from the state to make sure that these plans are actually implemented and not just shelved?

>> Barbara Flynn: Yes. Plans are designed to be three-year plans. What we're going to be doing is asking the districts for an update, partially because they have to provide some financial information to keep some of the grants flowing that they're incorporating in their plans.

>> Rochelle: Okay.

>> Barbara Flynn: And also to tell us where they're moving next, what they have accomplished and what the next areas are they're going to address. Because if it's a three-year process, hopefully they'll be moving forward, and it's not just going to be static and looking at the same things, but improving things each year. So we will be providing guidance in the summer on what they have to tell us and where they should go next.

>> Rochelle: The question then also arose as to what does the state intend to do with that information? Now that you have had it and you're reviewing back with the school district, what if the school district isn't meeting its goals? The question was will this be used against a district that for some reason runs into a problem in implementing its plan and can't do it... or how will the state use this information once they have it?

>> Barbara Flynn: Well, I don't think it will be used against the district. I think, rather, it's something that the staff can look at and say that this district needs help or maybe we can help somewhere or maybe we should get other help for the district through the BOCES district superintendent or someone else and see why the district isn't moving forward in the areas they have identified.

>> Rochelle: All right.

>> Dick Jones: In many ways, that broad-based planning committee in that district, that becomes the advocacy group for dealing with the problems, about where there may be a gap and why they didn't reach their goals, and to begin to address that within the district.

So for the most part, it's not going to be the state coming in and deciding whether they have passed or failed their particular achievement of their plan. We do have a monitoring process to monitor a few districts in the state, but for most, the assessment of that progress is going to be based within the district and it's their responsibility to follow through on that.

>> Rochelle: So overall it's really positive and really helping a district use its resources and put things in the right plan, so they can begin to plan and look at things on a long-term --

>> Dick Jones: The state has really become the facilitator of bringing people together on this. We have identified some districts that have done a good job in the planning process. They have been the leaders in helping to develop the training program. We just brought the players together and let this happen as a state initiative rather than it becoming a mandate from on high, saying what must be done.

>> Rochelle: Terrific. Thanks very much. We're out of time. We'll be back in a few moments with Comprehensive English Regents Review.

I want to thank Dick and Barb for coming back and joining us. We appreciate it. Thank you very much.

>> Dick Jones: Thank you.

>> Barbara Flynn: Thank you.

>> Rochelle: Stay with us. We'll be right back.

(Music)

>> Rochelle: Good evening. Welcome to "New York Learns, Teacher to Teacher." I'm your moderator, Rochelle Cassella.

The calendar says it's barely the beginning of February, but the thoughts of many in New York State, in the Department of Education and in classrooms around the state, are on the future, 20 weeks into the future, to be exact, when high school students in New York State will be sitting down to take the comprehensive English Regents exam.

"So what?" you might say. "It happens every June." But this year things are different. The revised and stricter English Regents exam now reflects recent changes made to the State's learning standards, and it includes some new elements. It also will be scored differently than past Regents exams, and all students in New York State, including those with limited English proficiency, must pass this Regents in order to graduate.

The comprehensive English Regents, scheduled to make its debut this spring, is the topic of our program. We'll do a quick review of the exam, its changes and what students can expect before we look at how some schools are teaching to those new standards and how some school districts and teachers, as well as the students, are learning from the tests themselves.

Finally, we'll talk about how these exams will be scored by taking an in-depth look at how two teachers reviewed one particular task from the sample test and visit another district working to train teachers on the tasks and the rubrics that will be used to grade students' work.

My guests for our discussion of the comprehensive English Regents: Jackie Marino of the State Education Department; from the Burnt Hills/Balston Lake School District, Jim Schultz, Language Arts Coordinator, and teacher Carol Reynolds; and from the Niskayuna School District, Co-Director of English Language Arts is Lillian Turner, and also Fannie Mack, Assistant Principal, English, at the A. Phillip Randolph High School in New York City.

Thanks, everyone, for joining us on this important day in this important discussion. We're going to start with Jackie for a brief overview of the exam and what students are going to be required to do. Jackie?

>> Jackie Marino: Sure. As you hinted earlier, Rochelle, the answer to that question really begins with the English Language Arts standards. The standards were developed by teachers a number of years ago and reviewed by the public, and so they become the focus of the new assessment.

We have some key statements from those standards. Standard one: Language for information and understanding. Just quickly, to give you a suggestion of what kinds of things people have agreed that students should know and be able to do before they graduate, certainly interpreting and analyzing complex texts, being able to extract information from those texts, making decisions about their value and significance, being able to write on a variety of topics in a variety of forms, and when they do that, to be able to present a controlling idea, use a variety of organizational patterns, support their interpretations and, of course, do that in the standards of written English. Those are all highlights of language for information and understanding and some of the behaviors associated with that standard.

And then there's language for literary response. Some of the key statements from that standard include being able to read many genre, identifying the distinguishing features of those genre and using those features to interpret the work, to be able to recognize literary elements and use those to interpret the work, to understand the different levels of meaning in literature and, again, to write about those using standard English.

Then the third standard that is assessed on this examination is Standard 3, language for critical analysis and evaluation. And some of these key statements include being able to, again, analyze and evaluate a wide range of texts and to evaluate those texts from a variety of perspectives as well as to understand the perspective of the writer that you are looking at, and to present those analyses in a well-developed way, in a well-organized way, using effective details and evidence from the text. And, again, to convey those ideas in standard English in a skillful way.

So that said, that suggests that if we want our students to be able to do those things, then an assessment of those things is going to have to ask them to do this kind of a thing.

And we have a slide of the four parts of the exam that were drawn from these standards. In Part 1 of the exam, we do give students a passage to listen to and they will see when they hear it, it will have in front of them the task that they're required to do. So they're listening for a particular audience and a particular purpose. It's different from the audience and purpose of the original speech. So it involves some transformation for interpretation. To help them do that, they'll get some multiple-choice questions that will not only assess their ability to comprehend those ideas, but they may give some hints as to the kinds of things they might write about in their written response.

And then in Part 2, which also assesses information and understanding, but this time in written form, they're given a text and a related visual. This is the kind of text that you might find in a public magazine or newspaper or brought into a content-area class to support the curriculum. In this case, the students need to write, again, a response for a particular purpose and a particular audience that may be different from the original. And they need to synthesize the information from both of those sources and, again, they'll have some multiple choice questions that not only get at basic understanding, but they may -- close attention to those questions may help them write their written response as well.

>> Rochelle: And the visual would be a graph or a chart?

>> Jackie Marino: It could be a graph; it could be a chart; it could be a diagram or an illustration, drawings of some kind, a map... something that presents similar but not identical information, supplementary but not identical information to what's found in the text.

>> Rochelle: Okay.

>> Jackie Marino: And then in Part 4 of the test -- sorry, Part 3. Part 3 of the test, assessing language for literary response, and in this case, the students are given two literary pieces. They'll be in different genre. They might get a poem and a part of a play script or they might get a memoir and a short excerpt from a short story, but two different genre. And they're told what the pairing might be in either text that you might read for many purposes. But in this case, we're asking them to look at a particular concept that is shared by those two pieces and asked to write a controlling idea that unites the pieces and then to use literary elements and literary techniques, the author's craft.

Again, they'll have some multiple-choice questions that can be helpful in sorting out what the texts mean as well as assessing vocabulary and basic understanding.

And then in Part 4 of the test, this section assesses language for critical analysis and allows students to talk about some of the texts they have read. This is in class, to distinguish from the texts that they're given on the test. And in this case, the students write an essay in which they're given a critical lens through which they will look at those two texts. We'll hear more about that when we see the clips from Carol and Lillian. So they write -- they're responding to works that they've read in class, in a critical way. So this will assess their progress toward those standards.

So that said, the first administration is, as you indicated, this June. And we've got a lot of basic information here. It will be offered in two three-hour sessions, the mornings of June 18th and June 23rd, divided between two tasks per session. In Session 1, on the morning of the 18th, students will have Parts 1 and 2, both informational in nature. Then the morning of June 23rd, they'll have the literary tasks.

And as we have always done, these tests will be offered in June, August and January, so students will have many opportunities to retake the test.

>> Rochelle: Um-hmm.

>> Jackie Marino: A lot of people are asking about the passing score, and while we don't yet know what numbers will correspond to those passing scores, because that will take place in the standard-setting process that I'll mention a little bit later, but we do know that the passing score to get a Regents diploma is 65, as it has been in the past. But during this transitional year -- these transitional years between the entering class of 1996/97 entering their freshman year and before they enter their freshman year in the year 2000, if the local board approves of that, they may count a 55 on this test as a passing score for local credit. But it's a transitional safety net with board approval.

That's the basics.

>> Rochelle: A lot of writing, a lot more writing this time around.

As we said, the tasks on the Regent exam are directly related to the learning standard recently adopted by the New York State Board of Regents. So it stands to reason that the best way to prepare students for the comprehensive English Regents exam is to teach to the standards.

Maureen Sarah teaches tenth and eleventh grade English in the small Minerva School District. When we visited her tenth grade class last spring, students were using surveys and journals to create pie charts to map their reactions to the literature they had read during the school year.

>> Okay. So does anyone want to start?

>> My favorite book that I read this year from English is "Angel's Ashes," which is my independent reading book that I picked myself.

>> I think the standards overlap in many areas. I see teachers, whether they're conscious of it or not, addressing standards in their classrooms, the same standards that I have for communication, social interaction, data, information and understanding. Why not connect those? Make it relevant for the student. If the student is graphing in my course and they're graphing in math as well, we can share that.

>> It was not interesting or enjoyable...

>> I think the new standards and the new tests asks students to be active in class rather than passive, to be responsible for their own learning, to be engaged in their own connections. I don't know that a test can raise standards, but I think that good teaching can raise standards, and by changing the way the test is written, it will ask teachers to call on students to use higher-level thinking skills.

>> So here's your task. Number one: Use existing data to investigate your controlling idea and create a graph or graphs which illustrate your findings.

>> I want to address the standards; I want to design parallel paths for my kids. I want them to feel engaged; I want them to feel they have some ownership, and I want them to reflect: What did I learn? What do I know now that I didn't know before? To reflect on an entire year's worth of literature, the graphing and using technology at the same time is a tad overwhelming, and I'm demanding that they be reflective about it. So I'm asking a lot, but I think that if the students walk away and say, "I know what kind of reader I am. I can make a recommendation to Mrs. Sarah based on my own interests," that's great.

>> Characters that you relate to, emotions that you...

>> Rochelle: That's how one teacher is handling things. For other looks at how other places are doing, I'm going to turn to Jim Schultz and Carol Reynolds from Burnt Hills/Balston Lake. What does it mean in your school to teach to the ELA standards?

>> Jim: It's really important to take a look at some of our classrooms and see what is happening in them. I'm really excited that you would need to go down to the kindergarten/first grade level to start, that you could walk into a first grade classroom and find the teacher asking students to make comparisons between two books she had just read aloud during the week, that we're looking at the standards at that level. You could move up to a fourth grade classroom and find one of our teachers dealing with the Civil War, as she always has, but looking at a fiction piece and then putting against it a non-fiction piece about the Civil War and asking her students to draw conclusions; perhaps do a graphic organizer that shows similarities between historical fiction and the non-fiction texts.

I'm really excited because we really seem to be scaffolding the skills for the kids all the way up. So when you go up to the middle school, you can walk into a sixth grade classroom and find them doing Gary Paulsen, because kids just love Paulsen, and taking a look at a text and then putting a poem next to that text and asking the kids to draw conclusions or similarities between the two texts.

You could peek in at one of our eighth grade teachers who would probably be reading an acceptance speech from someone as an introduction to a unit of that person's novel, so that they are practicing their listening skills and that is embedded into the unit, as kids have been doing all along.

As you moved up through the high school, as you went from room to room, you would see the variety of experiences that are available to the kids because the standards are so wide and ask our kids to be independent for us, for us to be providing experiences that aren't formulaic, that don't teach to a tests but to a higher standard.

And so in our ninth grade classroom, where we're doing "Romeo and Juliet," you might see a teacher doing clips from a Lerner film placed next to a clip from the Zeffirelli film next to a clip from a theatrical production film of "Romeo and Juliet." And the kids stand back from it and do that higher critical thinking that the standards are trying to elicit from the kids.

As you move into a tenth grade class, you could see the variety of writing tasks that we're doing in the writers' workshop classes. Look in one class where they're doing a career unit and they start by looking at graphs, projections, things that are important to them... what are the careers that will be there in ten years? What are the salaries and the top careers for the 21st century? I think the standards are moving the kids into the 21st century in that way. They look at the unit, research, jobs available to them. They set up a budget, as they will have to in a few years as they're out on their own.

>> Rochelle: (Chuckling) Oh, yeah!

>> Jim: And make decisions about money and reference charts that are there. They have to do a consumer unit where they go out and buy something for the apartment that they're planning, using all the skills of reading for information. As you move into some of our other electives, you'll constantly see teachers giving choice to students about the kinds of writings that they do, so they come out with, again, the wide variety that I was talking about. They're writing editorials; they're writing letters; they're writing critical analysis.

>> Rochelle: Carol, what are you seeing in your classroom?

>> Carol: In my own classroom, I find myself as a classroom teacher probably doing more with the first two tasks of listening and reading and writing for information and understanding. I don't think I'm unique in having focused more on literature and literary response and analysis in the past. I think the high school teacher has kind of seen that as her job both in terms of content material and in the response to the content material.

I think most English teachers feel more confident when they look at Task 3 and Task 4 on the new Regents because, although they are different and more sophisticated than what students have been doing in the past, they're closer to the kinds of things they're more used to doing.

So I find myself working to integrate more of the skills that involve -- more listening certainly and more of the reading and writing for information and understanding so that when the students do elicit -- if we do something in the classroom for listening, because it is also my job to help the students learn those skills, I'll suggest a graphics organizer, for example, that they could use to take notes as we listen to a passage and try to get the students focused on what they need to do to be successful in these areas where they probably haven't had as much experience.

Much more with visuals. I teach an American Experience course. We were discussing the President's Commission on Race in the fall, had an article from the paper, and I gave them the chart first and I said, "Take the chart and read the chart and jot down what you have learned from that." Whereas in the past, I might have given them the whole -- the article and the chart together and said, "Okay. Let's read this and then check off the chart." I'm much more focused on teaching the chart and seeing what you can learn from it and how you can integrate that information.

I think you'll see not just in my classroom but in others, you'll see students who actually have -- if they do a Regents task, and we have asked our teachers to do one of each of the four tasks each semester with every student. So you have students working with the rubrics, so they know and they are learning themselves what the five areas are, the qualities on which they'll be evaluated. They're using -- one of our teachers has given the students an essay and then given them the rubrics and they had to find two other students in the class to respond to their paper and assess it.

You'll see quotes on the board for students to respond to in their journals, to help lead them to dealing with the kind of critical language that they see in Task 4.

>> Rochelle: Lillian and Fannie, I'm going to bring you into the conversation as well. Lillian, what are you seeing in your school?

>> Lillian: Similar things to what Jim and Carol described. I think it's important to help students develop habits of mind that are going to serve them well in life as well as on the tests. Getting them used to thinking about connections, the way things link together, I think is certainly something that expands and enriches their understanding of concepts, and it will also have a very practical value for them in taking the test.

I'm thinking, for example, of collateral reading, that in my case, my students have done three different non-fiction texts, for example, and then coming together and creating Venn diagrams where they are looking for and noting the similarities between all three texts or among two of the three, around the circle. They're using the graphic organizer to help them articulate what that understanding is, and they begin then to get used to looking for those connections.

I think the kinds of things that are not clones of the actual test questions but require smaller activities that will support them in doing either a full-fledged clone or the exams themselves. I think we heard some good examples of that in terms of the interesting and I would say greater activities generally regarding listening activities that students are being involved with. We've heard some good ones already.

>> Rochelle: Fannie, what's going on at Randall?

>> Fannie: One of the big problems with anything new, and this test is no exception, is how much are we going to have to discard of what we're already doing and how much are we going to have to relearn?

So for us, the functional word is "focus," and in the various classrooms at Randolph, you will find teachers looking for ways to incorporate the new standards into the work that we already do. How can we pull the quotes, as Carol has said, from the literary works that we already teach but when we have not been focusing as much upon the quotes? We have found that our students really need an opportunity to learn to look at a quote and put an interpretation with it and then to write about it.

So we're moving in that direction, trying to pull as many new items as we can into what we're already doing.

>> Rochelle: Okay. A lot of things going on! Clearly understanding the Regents exams, the tasks that are called for and how the student answers will be scored can provide teachers with some valuable information that they can use now in the classroom to help prepare students. So we're going to talk about the samplers for a moment, and for that, I'm going to ask Jackie to give us an idea of what the samplers are like.

>> Jackie Marino: We do have a front page of the sampler, just in case you think they might have been in your school and you threw them out. (Chuckling) These were distributed to schools last June. You'll find in there a question-and-answer section dealing with the most frequently asked questions about the new tests as well as some sample tasks of each sort, and each of those tasks includes the scoring rubric used to evaluate student responses as well as some anchor papers related to that. We're hopeful that what schools will be doing with those, and I know many of them began last year when they received these things, is to give them to students, to have the students themselves talk about their reactions to the tasks; look at things like which of the tasks will be easier or more difficult for this student? How well are they using the notetaking and the planning pages?

When I went through hundreds of responses to the field tests, I was rather surprised by how few of those pages actually had any writing on them, so that certainly has an instructional implication right there, that people aren't making the best use of those pages that they have.

Looking to see if they're following the guidelines, because the guidelines repeat key words from the standards and anticipate the scoring rubric, so those are important. Doing things like asking students what experiences do they feel they need or wish they had had in order to prepare them for this kind of test?

I think it's important to remember that the test was developed in order to both enhance and reflect good practice so it's not surprising that the kinds of seamless activities that these folks had mentioned are going to be the ones that help students be successful on that test. The tasks are just supposed to be proxies of good practice; they're just supposed to be one moment -- rather six hours -- (Laughter) -- in time along the way, learning to use language in meaningful ways.

So those are some of the things that we hope people will be doing with the sampler in order to help prepare their students.

>> Rochelle: As we hear back from people, we've heard a lot of what's going on, and when Fannie was talking about "What do we have? What are we using? What can we incorporate and continue to use?," we're finding a lot of that going on, that teachers are saying, "Well, in a way I have been kind of doing some things; I just have to maybe enhance some of what I have been doing."

>> Jackie Marino: I should say, Rochelle, that the sampler, minus the student responses, is on the State Ed web site, and we have that address.

>> Rochelle: Okay. Wwww.nysed.gov, right. So in case you misplaced yours, you can go get another one.

What we're finding is that a lot of the samplers have been used for training, and we asked Carol and Lillian to conduct a mini-workshop for us to give us a sense of how they recommend that teachers use these samplers to teach their students. They worked on Task 4, based on Learning Standard 3, reading and writing for critical analysis and evaluation. In the first segment, they closely analyze the elements of the task and relate it to classroom activity.

>> Carol, we have our last workshop section coming up, and we need to prepare to do Part 4 of the new Regents. It's really the most interesting, I think, or in some ways it is because it's the only part that depends upon the class readings that students do as opposed to texts that are provided to students on the test itself. It's maybe also the one that teachers feel most comfortable with, do you think?

>> I think so, and I think they're going to see it as being very familiar, and if they don't look closely at the task, they're going to miss some of the implications of the complexity of it. I get worried that they're going to see it as being very parallel to the old Part 2 on the Regents --

>> Yes.

>> And if they don't examine the task themselves and get their students to do that and move to a different level of thinking, the students simply aren't going to be that successful.

>> They're not going to be ready for what this particular task asks for.

>> It does still ask for the two same, you know, two pieces of literature and it gives students a place to start but a very different kind of place, and we have to make teachers understand that.

>> To understand that, and also to connect their thinking in terms of the task with the standards.

>> Right.

>> That it's not simply a thematic essay that students are being asked for.

>> Right. I think we have to keep reminding people to go back, that it's Standard 3, reading and writing for critical analysis and evaluation. The one thing this task asks us to do is that the students themselves have to establish the criteria, and that's a very sophisticated kind of thinking.

>> This is what the students will see, or something very much like what the students will see for the actual test day. The students are asked to write a critical essay in which they discuss two works of literature they have read from the particular perspective of the statement that is provided in the critical lens, and in the essay provide a valid interpretation of the statement, agree or disagree with the statement as you have interpreted it, and support your opinion using specific references to appropriate literary elements from the two works. You may use scrap paper to plan your response. The kids are being asked to do quite a bit here.

>> There are so many steps in this particular part.

>> Yes, um-hmm.

>> Beginning with having to interpret the statements. I have found with my own students and with so many of the papers I have looked at that students will restate in their own words the statement, what it is, and not interpret it. And that's the point, one of the key points, I think, for students. It's in the interpretation that they can then establish and develop and clarify their own criteria to use to move on to the rest of the essay.

>> They need to think critically about it, too, in terms of whether or not they agree or disagree with it because that is going to provide the direction for the essay that they will compose. This is asking them as well to refer to appropriate literary elements, and that's probably going to be one of the challenges for students.

> It's been the challenge in papers we have seen already.

>> Yeah.

>> In terms of first knowing what that means. That certainly has classroom implication --

>> Absolutely.

>> We can talk to students about that. But then once you find -- once you're into the literary element world, tying that in carefully with this and not just saying, "Oh, and yes, there are metaphors..."

>> But making it part of the essay.

>> Yes, and part of the development of that essay.

>> Teaching students then how to read the question and to underline those parts of the question that are telling them exactly what to do will be helpful. I think also maybe teaching -- I think you alluded to this -- things like graphic organizers that will point up the literary craft, the literary elements of one of the cells in that graphic element-- like a chart, for example.

>> The simplest thing, and I saw teachers do it in some training, was just to make their own T-chart. Look at the question and say, "I need to do this; I need to do this; I need to do this," and then parallel it in the T-chart.

>> Word for word.

>> Right. So that works. And teaching students to use the bullets. In competency testing in the past, I've said to the students, "The bullets are the teacher's answer sheet. That's all we get as an answer sheet. So you have the answer sheet in front of you when you write your essay. So make sure you provide a valid interpretation. Make sure you..." The questions are constructed, as they are on the other tasks, so that the bullets reinforce the language of the tasks so that the student in some tasks is told three times what to do, so he or she should pay attention.

>> Yes. I have mentioned that in some training for scoring, that to us it might seem redundant, but if students are told two or three times what to do, this is helpful to students because it really does put them right on track for where they need to go with that essay. This is an example of a critical lens statement that students will need to work with, and it's essays based on that particular segment that we have been using and the scoring frame and the practice for that. But they're always different. Each administration of the test will use a different critical lens. So teachers might do a good thing by collecting some of these for themselves to use in the classroom.

>> Right. And in terms of classroom implication, students are not even used to taking a quote and -- they think restating it in their own words is a big deal because they're not used to even doing that. We have a number of teachers in our building who are putting a quote on the board, say once a week when they come into class, and it's there and the first activity of the day is to simply come in, sit down, copy the quote down and then in your journal interpret what that means to you, take it from its own words, put it in your words, and then talk about what it means and how you feel about it, just to get kids used to that initial piece. And that doesn't have to be necessarily about literature.

>> That's right.

>> It can be about the themes of what's going on in your classroom, that sort of thing.

>> This really does depend on the students' interpretations. It's not simply a task. They need to look at that piece of text, understand it, talk about it, and then pursue it according to their own life as opposed to the much more formulaic type of writing --

>> Right. Without that conventional definition of how setting impacts on characters that you might have encountered in Task 2. Now the students set those criteria and work within them.

>> Different level of thinking.

>> Right. Very different, very different.

>> Shall we look at the guidelines, get to those and see what implications they have?

>> Sure. First, essentially, it's the valid interpretation of the lens that establishes the criteria. So the students know right off that's what they have to do in order to get started. The second step is indicating whether you agree or disagree with the statement. I have teachers who have trouble with that step because that can feel artificial. You know, "William Faulkner says this, and I agree with William Faulkner"... That can be done in more or less sophisticated ways, but the student has to know that he or she has to take a stand on that idea.

>> Um-hmm. And then they can choose two works they have read that they believe best supports their opinions. This is also kind of wide open because those can be any two works they have read, from any genre, long works, full-length works, short works... it could be a poem and a short story that students might choose. They don't have to be full-length texts, necessarily.

>> Right. I think probably the reality will be that the richness of a full-length text and the time in class having been devoted to that will lead them to use that, but students need to know that they can use any kind of material in order to do that, that best supports their opinion. And actually, sometimes a poem might be easier to use in terms of the literary elements later, because the form of that is so conspicuous to the students.

>> Exactly. Absolutely.

And they use the criteria suggested by the critical lens to analyze the works chosen. Now they need to go back to their interpretation to figure out what those criteria are, then apply those criteria to the works, make sure the works fit.

>> Right.

And to me, that one talks about -- it says that's what you're going to deal with in the work and it's continued in the next. You're not writing a flat summary, but you're using specific references to the elements, the literary elements, and so some students are going to end up -- part of what you can do in some places here is to use the elements as the focus for the references from the text that you're including, or they can be paralleled to develop the analysis.

>> And I guess we need to point out, too, these are only examples of elements students might use, and we need to tell them that as they practice and prepare, that they don't need to be limited to those terms but, in fact, have the entire world of literary terminology at their disposal for completing that part of the expectation.

>> The last three should be very familiar. Organize your ideas in a unified and coherent manner; I think that's important for students to see what the test looks like, to get used to the language, that it's going to say "guidelines," "critical lens," to get used to that language, and then there are other implications that we can talk about for different kinds of activities in the classroom. I think the important thing about this new test is that it's not just that we're changing what the final exam or midterm looks like, but that we're changing the nature of some of the classroom activities with literature and text and writing.

>> Um-hmm. Enriching them, probably.

>> I think so. I think that's the goal.

>> I think that's one of the ways in which this new exam does enhance what goes on in the classroom. It's -- teaching to this exam is a good thing to do.

>> Rochelle: Terrific discussion, and it really included a great many excellent tips for teachers to use in the classroom and some real specific instructions with regard to both the exam itself and then also for general classroom instruction. Carol and Lillian, can you again kind of point some of those out for us, for those teachers who were not taking notes?

>> Lillian: Students have to know how to read the question. They have to learn what the question is asking for. They have to learn how to use a pen or a pencil to underline those key words that really direct them in terms of what they have to do. One of the ways they can learn that is to see lots of examples of those questions. When Carol said that they need to know what the test look like, what the format looks like, one of the ways we can do that, I think, is by fashioning tasks such as these, using literature the kids are reading in our classes as essay questions, for example, homework assignments.

>> Rochelle: And something that Carol brought out earlier was putting quotes on the board --

>> Lillian: Yes, that's a wonderful idea.

>> Rochelle: The critical lens...because I think that's probably really tough for kids.

>> Lillian: It's a continual balance for me in terms of familiarizing the students with the format of the exam, so they learn the language; they know what "critical lens" means; they know that they will have a task with a critical lens and then the guidelines. So they're familiar with the format, and then at the same time, just tying in all the richness of what it's asking the students to do in the classroom, so that they are seeing that they're establishing criteria; they are learning organizational patterns; they are doing those things as well.

>> Rochelle: Okay. Are teachers -- are we seeing teachers having this kind of dialogue, the kind of discussion that you had here in the studio and videotape? Are we seeing that going on in schools, Jim?

>> Jim: I think so. One of the important things that's happening is that those discussions are happening in different places, that in our department we'll have a department meeting and have that discussion around a training session. But in addition, our Regional Teachers' Center has been doing a phenomenal job of sending Carol, for example, up through districts around the State, around our area, to have those same discussions. Not just the format but what are some of the skills and how do all these things tie back to the standards?

>> Rochelle: Okay. What other kinds of things can we learn from a review like this? Once again we talked about making sure kids understand the definition, using critical lens. What other kinds of things can we learn from these kinds of reviews? Carol?

>> Carol: Well, I guess one of the things we're learning is exactly that we do have to go and educate the public teachers. If teachers themselves are not familiar with the tasks, then they're going to have a hard time -- the default value will be to train the kids just to do the tasks and not integrate it into the classroom, and Lillian and I have done the training for BOCES in this area, and the more you reach out and get to more teachers, I think the richer is going to be the experience for all of the kids in the classrooms.

>> Rochelle: Is this kind of staff development with regard to, you know -- we're doing a lot of training in talking with teachers now about -- in preparation for a transition in the exam, but is this kind of staff development something that really should be an ongoing kind of discussion?

>> Absolutely. It is. I think as we look at our curriculum and the ways we might want to adjust curriculum, that moves the discussion, certainly, to teachers of other grade levels. I think you're seeing as much discussion about the eleventh grade test among ninth and tenth grade teachers as you are among eleventh grade teachers. It should be an ongoing quality of the teacher's life. What I think the fact that the test is down the road has done is to make that really clear, and it's really jump-started a lot of those discussions, I think.

>> Rochelle: Fannie?

>> Fannie: What Lillian is saying, a large part of the staff development turns out to be teachers sharing with each other what they are doing within the classroom, so one teacher brings this graphic organizer that he is using to help his students analyze a piece of literature, and that becomes something that several of us within the department take on and begin to use, on his suggestion. So the staff development at this level is very important and it is -- appears to me much more teacher-centered than it has been in the past.

>> Rochelle: Teachers are training each other.

>> Jim: What Fannie says about teachers having hands-on in this aspect, it's very important. When I watch Lillian and Carol doing their training, and they take the task and ask the teachers to answer the task first so that they understand the decisions that the students will have to be making... that kind of hands-on understanding goes a long way in terms of assimilating what the test is about.

>> Jackie Marino: And people are asking, too, for help in some areas that they feel a need for. Some of the Teacher Centers are asking what is the most helpful professional development they can offer, and some teachers have said they wish they knew more about how to use these literary elements and how to help kids recognize the need to interpret literature, things like learning from listening experiences, how to do more of the embedding type of listening rather than just tacking them on at the end, making these tasks be a lay-on of their curriculum as it already exists, but how to help students to learn from listening and to learn from non-fiction. Things like terms, "controlling idea," not only what does that mean, but how do you help students to unite the idea from several pieces of literature and provide a synthesis that does justice to them all?

So there are many implications that have already been tapped by teachers who are looking for information and by teacher centers and other resources that are looking to provide it.

>> Rochelle: Okay. Let's move on to scoring a little bit because these are also -- the scoring is also going to be different than it has been done in the past. Jackie, what's changing?

>> Jackie Marino: Probably rule number one, as you score these exams, and also for the grade four and grade eight, is to put aside your preconceived notions about what good work looks like. Certainly some of that you'll draw on, but the important thing when scoring reliably is to always be putting the evidence in the student work up against the criteria in the rubric. That's what the trainers spend a lot of time on, providing an orientation to scoring.

It's important that teachers be very familiar with the tasks and the rubrics, and that's the reason for the sessions that BOCES have been providing or will be doing soon, to just help people be very familiar with them. And certainly your teachers, who have been using those rubrics for several years, in their various incarnations, and your students who have been seeing them, they'll be well prepared to know how to write to the task as well as how to score them.

We have found that we have had teachers come in three times now in the last year and a half to score various pretest results, and we have found that the more experienced scorers, not only do they become more reliable, but they drastically reduce the time they need to evaluate each text because they know what to expect; they know what the rubric says already. So we urge people to attend scorer training. We expect those workshops will be offered soon if they haven't been already in your region.

We also hope people will use the scoring materials that they will get with the exam or with that package of materials. When you come to score the exams in June, you'll have the rubrics, the final, refined version. We don't anticipate many, if any, changes on that, but we want to make sure it's as well stated as possible. You will be getting anchor papers for each score level, so you have some models to put your student papers up against. You'll also be getting a few practice papers for your -- we'll have a chance to kind of duplicate what teachers went through during the training on scoring day. I know time is limited to do that, but it's important to spend that time.

And, in addition, you will be getting the directions for the specific procedures to go through to determine the final score. And I know people are impatient for that and understandably so, but we don't want to make decisions without the data to support them. We know some things, however. We do know that you will be asked to record a holistic score only. It's been important in the training for teachers to look at all of the sub-scores, the analytic scores based on meaning and development and language use and so forth, and we've asked them to do that so they can become disciplined about what each of those qualities means. But in the final analysis, they'll be recording a holistic score.

We'll also be taking to the standard-setting committee two recommendations from the advisory committee. One is that each task -- there are four tasks -- be worth roughly the same amount, roughly equal in value. And the recommendation that the multiple choice/constructed response ratio will be roughly 25 to 75, so that the constructed responses -- it's been recommended that they be worth about three times what the multiple choice items are.

As I say, we're taking those recommendations to the standard-setting committee. We have to be sure that the field test results bear out those recommendations. But they'll be using those recommendations plus the field test data to help set the standards and decide what kind of score is equated, what work is equated with a 65%, 55% and so forth. So that committee will be coming in in April.

There's also the recommendation that we have three teachers score the exam so that no one teacher's score makes up for the total. And we know that means that in some very small schools, that may mean getting together with another area school so that that work can be divided up.

So you will be getting procedures eventually, and we have a number of knowledgeable people working on the best way to do that. It's always a balance between what's ideal and what's practical, what's workable?

>> Rochelle: Okay.

>> Jackie Marino: And then, of course, finally, we urge people to allow time both for the training up front beforehand and the scoring itself and to allow time the morning of scoring. Remember that no matter how much time you have spent on these rubrics and the tasks in the sampler and your own tasks, it doesn't really prepare you to score the tasks on the test until you have tried them out yourselves, have seen the anchor papers for those tasks, and you've scored some practice papers as well. So you'll be getting materials to help you do that and procedures to help you do that.

>> Rochelle: Okay. When we have them, we'll pass them along.

One of the things that practice scoring sessions can also do is provide teachers with tips for classroom teaching. In this segment, Carol and Lillian score an actual student paper and they point out what teachers can learn from it.

>> Once students have written that essay, it's going to be evaluated against the rubric.

>> Right.

>> Once again, I think teachers have probably seen this rubric -- here it is for them. We wanted to show the whole rubric. Even though you can't read it, we know you have this at home, and we are going to zoom in on the parts that are relevant. But what we wanted to point out as a reminder is that this rubric and the rubric for all of the parts -- one, two, three and four -- are essentially the same. That is they all are six-point rubrics, and they all deal with the same five characteristics or dimensions or criteria, however you want to put that.

There are always going to be meaning, development, organization, language use, and conventions, and the fact that the rubric is the same for all of those parts is really a very, very good thing. It says to students: These are the important qualities of all writing, not just for one part of the task or just for the test. I think teachers are beginning to use these rubrics daily in their classroom.

>> Right. Which is to the advantage of students.

>> Absolutely.

>> And to teachers in preparing their students. I think one of the things I remember from the very first part of training is to keep in mind that the entire rubric is a spectrum.

>> That's right.

>> And there's a line between each of the numbers, but there's not any, you know, solid brick wall.

>> Shall we read these?

>> Sure.

>> For meaning: "Meaning always means the extent to which the response exhibits sound understanding, interpretation and analysis of the task and text, and a high-scoring paper will provide an interpretation of the critical lens that is faithful to the complexity of the statement and clearly establishes the criteria for analysis and also uses the criteria to make an insightful analysis of the chosen text."

>> Right.

>> So that's the first bullet from the guidelines, essentially, and the combination, you know, bringing together the three things. Shall we talk through each of --

>> I suppose we can. Maybe we should look at the qualities and then look at the paper.

>> So then development, the extent to which ideas are elaborated using specific and relevant evidence from the text, this is the support, where the student puts the things together. "A six paper develops ideas clearly and fully, making effective use of a wide range of relevant and specific evidence and appropriate literary elements from both texts."

>> Those are two very fundamental qualities, characteristics.

>> Organization is the third of these criteria: "The extent to which the response exhibits direction, shape and coherence." We probably want to have some discussion about the meaning of those terms in our workshops --

>> Right.

>> -- for clarification. "The six paper maintains the focus established by the critical lens and exhibits a logical and coherent structure with skillful use of appropriate devices and transitions," which is an interesting place to focus on -- kind of large and internal organization in terms of dealing with students in the classroom, where the fluency of the piece comes into play.

>> And language use, the extent to which the response reveals an awareness of audience and purpose through the effective use of words, sentence structure, and sentence variety. The six paper is the sophisticated writer, essentially: Stylistically sophisticated, language is precise and engaging, notable sense of voice and awareness of audience and purpose, and the student -- the paper varies the structure and length of sentences to enhance the meaning.

>> And it's going to be very clear very early on in the paper as to whether or not it has these qualities certainly.

Okay. Conventions...

>> "The extent to which the response exhibits conventional spelling, punctuation, paragraphing, capitalization, grammar, and usage. The six paper demonstrates control of the conventions with essentially no errors, even with sophisticated language." I think people appreciate seeing that conventions line as part of the rubric. It does matter that students know these things and that shows up very clearly on the rubric.

So why don't we look at the paper we think we're going to use as our paper that's probably going to be pretty close to a six ultimately and see why it might be viewed in that way.

"In these politically correct times, it often seems that each individual's value system differs vastly from that of the man next to him. However, no matter how diverse two cultures, dogmas or individuals might be, there are some values, some truths that hold universal meaning for all. Great literature, according to Faulkner, calls attention to these all-encompassing, all-too-human themes of "love, honor, pride, compassion and sacrifice." I couldn't agree more. Literature which, regardless of its topic, strikes an emotional chord within all of us is the best. Great literature is literature that makes us think, makes us remember, makes us feel."

I think in that introductory paragraph, number one, we're clued in pretty quickly to the sophistication of the language use. But also in the latter part of that paragraph, the student does interpret the quote: "Literature which strikes an emotional chord is the best." And in doing that, he establishes his criteria: "Great literature makes us think, remember and feel." And so then he's going to go on to look at two pieces of literature and see how they meet the criteria.

>> Shall we continue?

>> Sure. "It takes the truly well-written novel to make us understand the emotions of an entirely alien situation. Once such novel is James Dickey's "Deliverance." Dickey calls on universal human emotions and experiences of fear, loss and determination to help us understand an experience unfamiliar to most of us: Pure survival. With his adrenalized prose, Dickey forces us to remember a time in our past when fear gripped us. With these shared emotions as his thread, Dickey weaves an emotional tapestry so rich that even the most extreme circumstances seem not only bearable, but believable and somehow familiar. Most of us had never been chased down the river by a homicidal madman. However, because most of us have at one time or another been afraid or have had to face an emotional or physical struggle, we can relate to much of what Dickey's characters are going through. Thanks to what Faulkner called "universal truths," we too are able to careen down a river, afloat on a raft of shared emotion."

See what I mean about the language?

>> Yes, this writer has control over sentence structure, with wonderful variety, wonderful rhythm to the prose, variety in length and in structure, and I think he's making it very clear that the stylistic element or the literary element that lets this or makes this book work in terms of the criteria he has established is the language that Dickey uses, and I think he captures that idea here.

>> And the student discusses it metaphorically with his tapestry metaphor.

>> Right.

>> So there's an example of the beginning of a really outstanding paper. He goes on to do just as well in the rest of it. And so, using the rubric, you can go across the rubric and point to the things that he does well.

>> Yes.

>> He's established the connections, has developed those connections. Certainly organization and language use and conventions are also in that realm.

>> We need to use that rubric to give credit to what the student does, to look at it as a credit model as opposed to a debit model, what he doesn't do.

>> Rochelle: The initial reaction in hearing a paper like that one, such as that one, you know, "the tapestry, floating on emotions," what happens when you look at that as a teacher scoring and says, "Oh, sounds great! It's a six. Let's move on to the next paper"? What happens in that kind of situation?

>> That's when you have to say what we say over and over again in training, which is we have to take the paper and go back and look at the rubric. Language use is one of five qualities. This paper does use language beautifully and he's very sophisticated in that. He also interprets the lens, gives us a good interpretation, establishes the criteria of good literature, making us think, remember and feel, and he puts the two pieces that he talks about against that. And so it's a six; I think we'll agree in that area.

In the area of development, he probably could have included -- we would probably be happier if the paper included more specific references to the text and maybe even if it talked a little more explicitly about elements. So it may not be a high six in that area, but it's still an excellent paper and we can point to those areas. It's not just that he overwhelmed us with his language but tied it to -- established the criteria and tied the pieces in, organized it very well. The focus remains throughout the paper and has very good control of conventions.

>> Rochelle: It also would have been very easy for a scorer from a different point of view to pick it up and say, "All show. Flashy language, sounds good, no substance. I don't like it."

>> So if you take it to the rubric, as Carol said, you're going to have to admit that that is a very positive quality of the paper, and yet on some other points such as she did point out, that acknowledges where the paper is somewhat wanting.

>> Rochelle: Okay. Which points out how important training is for teachers in the rubric.

Last August, the State Education Department conducted English Regents scoring training sessions for some 150 school representatives from BOCES and large school districts. Fannie Mack was one of those who attended the training sessions and has been working with teachers at A. Phillip Randolph High School in sharing that information.

Fannie, tell me a little bit about what your training sessions at A. Phillip Randolph are like.

>> Fannie: We have organized sessions around our department conferences. When I returned from the Albany session this summer, I consulted with the Director of Instruction of the Manhattan School District. Carol Bouvelle, and others of us who had attended the session, arranged to conduct training sessions for not only the Assistant Principals English but the Assistant Principals and Coordinators of Foreign Language, of E.S.L. and of Special Education classes within the Manhattan School District. They have been arranged at our monthly meetings.

Within my own school, I consulted with the principal and arranged to have extended time for our department conferences so that we could arrange to have a training session for each one of the tasks, as soon as we can get to them.

>> Rochelle: What you're saying is that each particular training session you focus on a different task? You focus on the rubric and you go through samples and...

>> Fannie: Yes. We first go through the rubric, just as Carol and Lillian did. We are trying to be sure that we understand the rubric, that we understand the differences in the rubric. For instance, I have a graphic that illustrates how the Task 3 rubric contains all the same elements as all the others, but it's different in that the purpose of the task is to assess the students' ability to read two works, to interpret those works using a common theme, to analyze the author's craft and then to write the essay according to Standard 2. So when we then look at the rubric for Task 3, we find the Task 3 rubric speaks to that particular purpose, in that it's different in the levels of comprehension and in the levels of development.

For instance, the student -- in order to do well, the student has to indicate that he understands both texts, that he has a clear understanding of the task and of two texts, because Task 3 deals with two texts. So he has to also understand the controlling idea and he has to understand the connection between the controlling idea, the two texts and the elements.

That is where Task 3 and the Task 3 rubric differs from the other rubrics. So we have to be sure -- that's the first thing that we do in our training sessions, to be sure that we understand the rubric as it applies to that task. Then we read the sample, and we're using basically the task that was sent in the sampler, the sampler draft that came in June of '98. So we look at the -- and there were two for most of the tasks; there were two examples. So we look at those examples and we ask each teacher to write his or her -- at least a beginning of an answer and their focus and which literary element they would focus upon and so forth.

>> Rochelle: All right. What we're going to basically do is we were able to take our cameras to Fannie's training session on scoring Task 3, and here's what it looked like.

>> Fannie: If we could now -- if there's no more discussion on this area -- move to the rubric. That's the first thing in the package that I passed out to you, the top sheet.

The purpose of today's session was to give us an opportunity to analyze Task 3, to look at the rubric for Task 3, to look at some specific literature, in this case a poem by Wilfred Owen and an excerpt from Tim O'Brien's "The Things They Carried," and to put all of those together in terms of how we would grade actual students' papers. And these were actual papers that were written by students in the State of New York.

>> "The poem describes the weather and how dark, cold and silent the night is..."

>> Fannie: We have seen the sampler and we've seen other sample tasks. Beyond that, we don't know anything else about the test, so this gives us an opportunity to visualize what might show up on the test, what types of things might be asked on the test, and most of all, it gives us a chance to think about what our students need to know, not only in terms of skills, but what type of literature do they need to have read, what types of writers?

>> Passage one, Wilfred Owen: "Our brains ache in the merciless iced east winds that knife us.

>> Fannie: So the closer the exams come, the greater is the tendency to panic. "Oh, we can't do it. We'll never do it."

>> As English teachers here in New York State, we really knew nothing about the new Regents, only that it was changing, was changing in this year. We were very familiar with the old Regents. I'm here at the training session to learn about the new Regents and hopefully that will help me with my students in the classroom.

>> So in the midst of this horror...

>> I think the whole department is very concerned about the new changes, and we want to find out as much as we can about teaching the kids and how we're going to grade these things.

>> He might have said that some of them, you know, were...

>> I think just doing this together is a big help because it stands out that we're not so far apart, and when there is a question, it really is helpful to work it out when we have the luxury of the time. When we're doing hundreds of them, we can't sit and look through it, but this way we can actually have the conversation.

>> We gave it an overall two, and we disagreed a little bit about the threes and twos.

>> Fannie: This is the third in our series of training sessions, and after each one, two things happen: Teachers are highly motivated to give sample tasks to their classes, and we always have an extended conversation about the skills we need to teach in order to raise our students' performance.

>> What's the meaning? What's the meaning? He's talking about the effect on many people...

>> I agree, that part, but for the task, we sort of understand...

>> Fannie: It gets teachers to thinking. Teachers will come in and say, "I found this article from the newspaper. I think it would make a good practice for Task 1." Or "I'm teaching this novel and I pulled this quote from it, and I think it would be a good idea to let the students get practice with Task 4."

There will be several teachers talking in the teachers' room about how -- the mistakes that we most often find students making when they practice Task 3 and what we can do about that. What are some techniques we can use to make the students understand they have to read the task carefully or they have to write more carefully? It creates a conversation among teachers about what we have to do, which is ultimately what we're about anyway. What do we have to do to bring students up to standards so they perform at and above the standards?

>> A two.

>> All right. That's your holistic score?

>> That's right.

>> Do you have -- can you tell me if any one of the qualities was different?

>> Between Roberta and I?

>> For me in the classroom, when I took what I found in the training sessions back to the classroom, I started grading students' papers on the same rubric that was in the training sessions, one to six. So they would get an idea of what a six was, what a five was, what a four was, trying to take that grade and convert it to a number grade that they could relate to. So they knew if they got a three on the Regents what that three was on a 1-to-100 scale.

Also, it's a lot more writing than the old Regents, so there's been a lot more writing in my classroom.

>> In my own classroom, I used to stick with a lot more literature and just discuss literature in the classroom and grade the papers grammatically on an as-needed basis. Now I find myself, this year for the first time, looking at the Regents a lot more, doing practice Regents, more practice writing assignments based on the tasks that I learned in the workshop -- Tasks 1, Task 2, Task 3 -- and I actually find myself actually doing more classes that are geared directly towards the Regents.

>> I think the test is going to be very confusing, but I don't think it's so different from what we have always looked for. We want kids to be able to read, understand, and be able to use their understanding to make connections, analyze what they're reading and communicate so someone else knows what they're talking about. And I think those are skills they need whatever they do. It's just in this case we have to make sure that we're all looking for the same thing.

>> Quite frankly, I use the rubric to help grade papers. I write the comments down that are on the answer scale -- I teach A.P. English, and I use the answers to help me grade my papers. Quite frankly, the kids see what they're not doing right and how they can correct them, to make revisions. It's quite helpful.

>> Vocabulary is sophisticated...

>> Fannie: This is the first administration. We have to learn; there's no other way. And we have moved our students to the point where they could score very well on the old Regents. We can do it for this Regents as well. It's going to take time. We have to have patience with ourselves and patience with our students and develop the skills as best we can.

>> Rochelle: Excellent advice! (Laughter)

>> Good idea.

>> Rochelle: Along with some of the training sessions last summer, Jackie, that you had for people like Fannie, what other kinds of preparation have teachers been having so far to help them become good scorers?

>> Jackie Marino: Some people I know have been following the progress of standards and the assessment for the last five or six years, and looking at the standards from the start, the very first pilot three years ago, the second pilot... And those people, of course, are not surprised to see what's on the test because they have been thinking about those things all along and struggling with us to make those refinements.

In addition to that and Teacher Center courses that have been offered in some places, we also have had, I think by now, probably over 400 teachers who have participated in scoring the field test results. Whenever those teachers have come together to do that, to do what is really an onerous task, they have worked hard, many long hours for several days at a time. They have uniformly felt as though that was a very valuable experience in professional development.

So scattered around the state are some 400 teachers who have had an opportunity to talk to each other about the implications of the assessment, to get the kinds of understanding that you saw in the video with Carol and Lillian, really thinking through what every task means and seeing a range of student responses across the state so that they can see what kinds of implications there are for their own teaching as well.

>> Rochelle: I want to go back to Fannie for a second. I know that you really -- this isn't just a matter of "We go through this session today and then we move on to the next task the next time," but your training really is ongoing and you have a whole booklet that you prepare for teachers and they come back with things to you. Tell us a little bit more about that.

>> Fannie: I have found that it's easier for us to deal with the various tasks if we approach them one at a time. So I have prepared a training manual for teachers and each department conference we approach one of the tasks, as we said before. So I bring in the materials color-coded --

>> Rochelle: Hold it up. I want everybody to see it.

>> Fannie: Just to make it interesting. We produce a packet of materials for each training session. And then after we have gone through that session and that task, I ask the teachers to then go back to the classroom, and that's where the real staff development takes place. The teachers then go to the classroom and look at what they are already doing and look at how this can be applied. That's when teachers come with various forms of a rubric that they are using, with blank forms as Carol talked about, and they talk about having their students develop the rubric for an essay assignment. So then we come back to the next meeting and we talk about what we have found and what we have learned based upon our study of the previous rubric, and I believe it has worked very well.

>> Rochelle: Okay. Lillian and Carol, other than regional, you know, training sessions, what else have you been doing and finding teachers doing to make themselves better scorers?

>> Well, the purpose of the regional scoring training was to prepare people to go back to their school, to run the kind of scoring training that Fannie has talked about. So in our school, we have done that. We have now had three early-release days and we have also looked at Tasks 1, 2 and 3 up until now. We have had half-days in order to do each one of those tasks.

And what I have asked teachers to do for each of them is to bring in some student papers that are written on that same task. We have eight teachers teaching eleventh grade English, and so they have been able -- two teachers each time have been able to bring in some of their students' work. After we have looked at the rubric and studied it and applied it to the exemplars and then to the practice papers that we have received from State Ed, we have ended that session by looking at our own students' work. That's helpful to us because it gives us a sense of our students in terms of this question and, you know, sort of culminating this really intense study of the rubric. But it's helpful for the students as well. Naturally, we're going to take them back to the class and talk with students.

>> Rochelle: Well, I know that one of the things that we also wanted to be careful of is that we talk a lot about using the samplers and learning from the samplers. But, Jim, you have a caution, in effect, that we should be careful about not relying on the samplers too much.

>> Jim: I think it's really important that as we're taking a look at the samplers, we not look at them as formulas for success. They're meant to inform the kind of discussion that Lillian was just talking about, and I think our greatest resource for staff development is really the expertise of our teachers. If we bring this back to our teachers and ask them, using their expertise, to go out and create these tasks, when they start searching for those sources, and that is a difficult task, they will start to understand the components of the task. When they put together -- put that next to the rubric and actually deliver it to their students, I think the understanding will be deeper than just taking a look at a particular task in a sampler.

Part of the standard speaks to the independence. So we learned from the fourth grade sampler, when we took a look at that, that when the actual test came along, it used very different genres, very different forms than in the sampler. And so we really need to be providing our students with a very rich variety of writing opportunities.

>> Rochelle: Absolutely. There was some discussion about the idea of when we're doing some training and we're talking about, you know, what are the definitions and the rubrics, what does this mean and this mean... is there any kind of discussion, anything that's gone around with a glossary for all the terminology so everybody interprets the words in the same way?

>> Jackie Marino: Actually, there is one. It's not in the sampler, but it is in the training material that teachers get when they go to these sessions.

>> Rochelle: Okay.

>> Fannie: I did include that as a part of my training manual in the very beginning, a list of the glossary, the test development process, all of these which help to inform us so that we know how we got where we are and we're able to look at it with a wider view and I think approach the task better.

>> Rochelle: Okay. Jackie, what if school districts want to develop their own assessments?

>> Jackie Marino: So that they'll be familiar with the format of these tasks and so forth?

>> Rochelle: Yes.

>> Jackie Marino: I just need to reiterate Jim's point, that the goal here is not to develop a curriculum around these four tasks. But having said that, we know people are eager that their students are familiar with them and want to embed these kinds of things in the curriculum along the way.

Teachers who have helped us develop the tasks have told me that the specifications that we give them for submitting tasks and texts have been helpful to them in this kind of endeavor. I put a few of these up on the screen. They get a lot of specifications for these, many of which are peculiar to State tests. We have a lot of constraints, of course, in State tests. But some that are particularly relevant to local schools, our first in Task 1, of course, it's an informational task. In doing these, the student must rely on information from the text, and this is not a personal response. So when you're developing your own -- there are important places for personal response, but that's not what the students are scored on here.

Also, as I indicated before, these go beyond summary. Summary is important at the elementary level of the standards and at the intermediate level of the standards, but when you get to the commencement level, we're looking for application or transformation. So even if it looks like a summary, it usually is a summary for another purpose and another audience. So that's important to keep in mind.

And while at the State level probably significant and relevant turns out to be passages that seem innocuous to people, you are not constrained in that way in the local school, and when you're finding tasks and texts that you can weave into your curriculum, that's the best way to incorporate these. So what kinds of things would enrich the literature study? For example, what kinds of non-fiction texts would enrich that? What kinds of graphs and charts would enrich social studies and science and so forth? So those are just three.

And Task 2 is very similar, the same kinds of specifications for Task 2, except that with the addition that it requires a synthesis of two texts. I think it was Lillian who mentioned before that it's making connections, not just this task, this text and this graph, but making connections with what we learned this week with what we learned last week or this author and that poem or whatever, that proce