EMSC

Elementary, Middle, Secondary & Continuing Education

RFP #09-021

Title: Elementary, Intermediate (Middle) Level, and High School New York State English as a Second Language Achievement Test (NYSESLAT)

The New York State Education Department (NYSED) is seeking proposals for the New York State English as a Second Language Achievement Test (NYSESLAT) for test development, research, printing, shipping, scoring, and score reporting. Eligible applicants are testing companies and/or educational organizations with demonstrated experience in developing and administering large scale assessments, printing and shipping a high volume of tests and test related materials, preferably including audiotapes and/or CDs, in a timely and secure manner. Subcontracting will be limited to forty percent (40%) of the annual contract budget. NYSED encourages the use of Minority/Women-Owned Business enterprises as both prime contractors and subcontractors. Please see the section of this RFP for information regarding NYSED’s Minority/Women-Owned Business Enterprise Procurement Policy. One contract resulting from this RFP will be awarded for a term of five (5) years. The contract will begin on April 1, 2010 and end on March 31, 2015.

Proposals must address the test development, research, scoring, printing, and shipping of six (6) annual tests in grade level bands Kindergarten, 1-2, 3-4, 5-6, 7-8 and 9-12 with four (4) subtests (modalities) in Reading, Writing, Listening, and Speaking, linked to the State’s approved English Language Arts (ELA) and English as a Second Language (ESL) learning standards. The proposals must also include assembling one prior year NYSESLAT test to be used by schools as an initial identification test on an ongoing basis for the identification of students as limited English proficient/English language learners.

In New York State, limited English proficient/English language learners (LEP/ELLs) are students who by reason of foreign birth or ancestry speak a language other than English, and score below the proficient level on the current initial identification test, the Language Assessment Battery-Revised (LAB-R), or the NYSESLAT.

Bidders should refer to the Mandatory Requirements section of this RFP for all Mandatory Requirements and to the Schedule of Deliverables located in Section 5.) Submission Documents when preparing their proposals. Bidders must adhere to all Mandatory Requirements and perform all required deliverables.

Table of Contents

Components contained in RFP Proposal #09-021 are as follows:

  1. Description Of Services To Be Performed
  2. Submission
  3. Evaluation Criteria and Method of Award
  4. Assurances
  5. Submission Documents Word document icon (606 KB)

Questions regarding the request must be submitted by E-mail to EMSCNYSESLATRFPINFO@MAIL.NYSED.GOV no later than the close of business November 20, 2009.  Questions regarding this request should be identified as either Program or Fiscal.   A Questions and Answers Summary will be posted to http://www.emsc.nysed.gov/compcontracts/compcontracts.html no later than December 2, 2009.

Contact Information for Questions

Program Matters

Fiscal Matters
Alice D. King Gary Yorkshire
(518) 474-5902 (518) 408-1715
E-Mail: EMSCNYSESLATRFPINFO@MAIL.NYSED.GOV E-Mail: EMSCNYSESLATRFPINFO@MAIL.NYSED.GOV

Eight (8) copies of the technical proposal (one bearing an original signature in blue ink) AND three (3) copies of the cost proposal (one bearing an original signature in blue ink) AND one (1) electronic version in Microsoft Word (CD format) of the technical and cost proposal must be received no later than December 21, 2009 at 3:00 PM.

Send to:

New York State Education Department
Bureau of Fiscal Management
Contract Administration Unit
89 Washington Avenue, Room 505W EB
Albany, NY 12234

Facsimile copies of the proposals are NOT acceptable.


Submit in a sealed envelope labeled "Bid Proposal - RFP #09-021 Do Not Open"


Submit Cost Proposal in a separate sealed envelope labeled "Cost Proposal – RFP #09-021 Do Not Open"

Back to top

Description of Services to be Performed


Work Statement and Specifications

This section of the bid package details the services and products to be acquired. Please note that the contract process also includes general New York State administrative terms and conditions, as well as terms and conditions required by New York State law. These terms and conditions address issues related to both the submission of bids and any subsequent contract; they are included separately in this bid package for your information. Please review all terms and conditions.

Overview

The Federal Title III program of the No Child Left Behind (NCLB) Act of 2001 requires annual assessment of English proficiency of limited-English-proficient students. NCLB requires demonstrated annual improvement and adequate yearly progress for such students in order for them to develop English proficiency and meet challenging State academic content and student achievement standards.  New York State regulations also require an annual assessment of English proficiency of limited English proficient students using the New York State English as a Second Language Achievement Test (NYSESLAT).

To meet these requirements, the New York State Education Department (NYSED) is requesting proposals for the test development, research, scoring, printing, and shipping of six annual tests, with four modalities (Reading, Writing, Listening, and Speaking) aligned with the State's approved English Language Arts (ELA)/English as a Second Language (ESL) learning standards. These tests will be used to assess the English language proficiency of students in Kindergarten through Grade 12 who are limited English proficient (LEP)/English language learners (ELL).

There are approximately 220,000-260,000 English language learners enrolled in NYS schools. The tests are currently developed in five grade level bands: K-1, 2-4, 5-6, 7-8, 9-12, each in four modalities: Reading, Writing, Listening, and Speaking. After the revisitation of the learning standards, the NYSESLAT will be developed for six grade level bands: K, 1-2, 3-4, 5-6, 7-8, and 9-12, in the same four modalities. The test must be developed in accordance with the Standards for Educational and Psychological Testing(American Educational Research Association, 1999) and New York State Learning Standards and Core Curriculum. The test must be consistent with the principles of Universal Test Design and must also be consistent with applicable Federal and State testing requirements.

The purpose of the tests is to measure annual student improvement in achieving English language proficiency in order for students to ultimately exit LEP/ELL status and move into English mainstream programs. In addition, the vendor must assemble six similar tests at the same grade level bands and with the same four modalities from items selected from previously administered NYSESLAT administrations (2007-2010) for schools to use on an ongoing, on-demand basis to initially identify new students who may possibly be LEP/ELL.

Background

The New York State Board of Regents is responsible for setting educational policy, standards and rules. The Board has approved and published learning standards in seven curricular areas: Mathematics, Science and Technology; English Language Arts; the Arts; Languages Other Than English; Health, Physical Education and Family and Consumer Sciences; Social Studies; and Career Development and Occupational Studies. The Board has also approved and published learning standards in English as a Second Language. At this time, NYSED, under the guidance of the Board of Regents, is reviewing and revisiting the standards in English Language Arts and English as a Second Language. The exact timeline for adoption by the Board of Regents of these revised standards is not known at this time.

The Learning Standards are the foundation for a rigorous system of assessments designed to:

  • Evaluate higher-order thinking skills and performance abilities, including planning and acquiring resources, problem-solving, conducting independent research, and producing real world products.
  • Provide information that helps teachers adapt instruction to students' strengths and needs; and informs students, parents, educators and the general public about what students are expected to know and do.

The current learning standards are available at:
http://www.emsc.nysed.gov/ciai/cores.html
http://www.emsc.nysed.gov/biling/resource/ESL/standards.html.

The NYSESLAT was developed by NYSED in collaboration initially with Educational Testing Service and, more recently, with Harcourt Assessment, Inc., (now Pearson) to measure the annual progress of students who are LEP/ELL in achieving English proficiency and to make a determination about declassification from LEP/ELL status.

The initial identification test currently in use, the Language Assessment Battery-Revised (LAB-R), was developed by the New York City Board of Education, in collaboration with NYSED, in order to identify students as LEP/ELL and in need of ESL or bilingual services.

Mandatory Requirements

The vendor must agree to all of the following mandatory requirements as well as perform all of the deliverables as outlined in this RFP.

Mandatory Requirements - Submission:

  1. Any proprietary material considered confidential by the bidder must specifically be so identified, and the basis for such confidentiality must be specifically set forth in the proposal.
  2. Vendors must bid on all subtests and grade bands.
  3. The vendor must identify a program manager, with a Bachelor’s Degree or higher from an accredited college or university, to be the central point of contact with NYSED.  A copy of the project management certification, if applicable, must be included with the proposal. The vendor’s program manager for this contract must not have primary responsibility for any other contract with NYSED or a large-scale testing program in another state.
  4. The bidder must include with the submission of the technical proposal, at least three current professional references to substantiate qualifications. Current shall mean references relevant within the last three years.
  5. For those activities that will be subcontracted, subcontractor names, MWBE status, specific services, and costs must be specifically indicated on the Subcontracting Form located in the Submission Document section of this RFP.  NYSED reserves the right to approve all subcontractors. Subcontracting is defined as, "Non-employee direct personal services and related incidental expenses, including travel."
  6. All bidders must sign and return the Mandatory Requirements Certification located in 5.) Submission Documents. Proposals that do not include the signed Mandatory Requirements Certification will be disqualified and removed from further consideration.

Mandatory Requirements – Deliverables:

  1. The vendor must use the following resources when developing new NYSESLAT test forms, initial identification tests and examination related materials. Attachments A, B, and C are requirements for developing new NYSESLAT forms.
    1. Template for NYSESLAT Operational Test Blueprint (Attachment A)
    2. NYSESLAT Test Specifications by NYSED ESL Learning Standards (Attachment B)
    3. Test Configuration/Points Charts (Attachment C)
    4. Speaking and Writing Rubrics (Attachment D)
    5. ESL Learning Standards- http://www.emsc.nysed.gov/biling/resource/ESL/standards.html
    6. Core Curriculum Guide for English Language Arts - http://www.emsc.nysed.gov/ciai/ela/elarg.html
  2. The examinations must be developed in accordance with the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, and the National Council of Educational Measurement in Education, 1999).
  3. The examinations must be consistent with the U.S Department of Education’s Title III of the Elementary and Secondary Education Act of 1965 (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). Notice of proposed interpretations that appeared in the Federal Register/Vol.73, No. 86/ Friday, May 2, 2008.
  4. The examinations must be consistent with the principles of Universal Design for Learning. (http://www.cast.org/ external link icon & http://education.umn.edu/nceo/TopicAreas/UnivDesign/UnivDesignTopic.htm external link icon)
  5. All bidders must adhere to the specifications contained in NYSED’s Presentation and Format of Tests for Visually Impaired Students (Attachment E), specifications for Braille (Attachment F), and specifications for large type (Attachment G).
  6. Certified New York State teachers must be invited by the vendor to review passages, items, and test forms, and participate in range finding and standard setting.
  7. The vendor is responsible for all the arrangements and costs for teachers participating in test development activities, including meals and travel expenses. Teachers residing more than 30 miles from Albany must also be reimbursed for their lodging. Teachers from throughout New York State will be invited to participate and may travel distances as near as 3 miles or as far as 350 miles each way. For budgeting purposes, vendors should assume an average travel distance of 150 miles each way and assume that 90% of participating teachers will require lodging.
  8. All travel expenses for teachers are to be reimbursed by the vendor at the approved New York State rates. New York State rates are available at http://www.gsa.gov/perdiem external link icon and http://www.gsa.gov/mileage external link icon. All travel arrangements for vendor staff must also adhere to these rates.
  9. The vendor will also pay teachers honoraria of $175 a day for a 7.5 hour work day. The honorarium will not be payable for travel days.
  10. Vendors may propose to use previously used items provided those items are not published on any web site or in any textbook or review book. However, all items used, whether they have been previously used or are originally developed under the contract resulting from this RFP, must be aligned to New York State standards and must become the property of NYSED.
  11. As soon as practical after revised ELA/ESL standards and core curricula have been adopted by the Board of Regents, test specifications revisitation subcommittees will be arranged by the vendor in K-4, 5-8, and 9-12 grade bands for the review process. The vendor, as part of its test specifications revisitation, must work with NYS teachers to revise the grade bands for the tests to include a total of six (6) grade level bands as follows: K, 1-2, 3-4, 5-6, 7-8, and 9-12. Prior to this revisitation, the vendor will provide operational tests in five (5) grade bands as follows: K-1, 2-4, 5-6, 7-8, and 9-12.
  12. All activities proposed by the vendor must receive prior approval from NYSED. Twenty (20) business days must be provided by the vendor to NYSED for review of items, field test forms, operational test forms, scoring keys, and rating guides plus related deliverables. This will allow NYSED sufficient time to provide feedback to the vendor and give final approval of the deliverables.
  13. All draft technical manuals must be subject to careful and complete editing by the contractor’s editors and proofreaders before the vendor submits them to NYSED for review and approval.
  14. All vendor correspondence related to the NYSESLAT program, including that to schools and committee members, must be provided to NYSED for review and approval.
  15. All bidders must adhere to NYSED’s security protocols regarding transmission of secure materials via encrypted files and secure shipment of all materials in locked boxes, using a delivery service with Next Day delivery and that uses on-line package tracking. Electronic transfer via mail, Internet, or facsimile (FAX) of individual student information or secure materials is not permitted, unless authorized by the Director of State Assessment to do so on a case-by-case basis. The vendor must adhere to the Security Guidelines for the New York State Assessment Program (Attachment H).
  16. All vendor staff and subcontractors’ staff assigned to this project must sign a Non-Disclosure Agreement after the contract has been awarded (see Attachment I). Copies of all such signed agreements must be furnished to NYSED within 30 days of contract award. 
  17. All materials are to be held strictly confidential and must not be copied, duplicated, or disseminated by any manner or discussed with anyone other than persons authorized by NYSED.
  18. The vendor must develop and maintain a secure web file transfer protocol (FTP) site as a means of file transfer. Access to NYS test information on this site must be limited to the vendor and NYSED unless further sharing with other parties is authorized in writing by NYSED.
  19. The vendor agrees to cooperate with NYSED with respect to any challenge to the NYSESLAT examination or Initial Identification Test and, if necessary, provide at no cost to NYSED, experts, evidence, witness testimony, or other documentation necessary, within reason, to refute a challenge to the validity, reliability, cultural fairness, or any other aspect of the development of the examination with which the vendor was associated for a minimum of five years after a particular test item, test form, or operational test is administered.
  20. The contractor must provide a means for key staff to be reached during periods of field and operational testing.
  21. The vendor must provide a customer service number which is adequately staffed from 7:30 A.M. – 4:30 P.M. EST each business day to handle all NYSESLAT-related questions from school personnel during the ordering, administration, and scoring periods (January – June).
  22. The vendor shall reproduce, use, display, and include copies of NYSED’s trademarks, trade name, logos, copyrights and other intellectual property (collectively, the “Marks”) on all copies of materials produced for the NYSED. The vendor acknowledges that the Marks are owned solely and exclusively by NYSED, and nothing contained in the resulting contract shall give the vendor any ownership right or interest in such Marks.
  23. All invoices submitted for payment must include dates of services and an itemized list of activities and costs consistent with the approved schedule of deliverables contained in the executed contract. Payments will be made upon 100% completion of each deliverable.  Payment(s) contained in the invoice for subcontractor(s) must list the subcontractors’ names, payment amount, and nature of services provided separately on the invoice submitted.
  24. The vendor must retain and update records and accounts on a monthly basis and must be able to prepare and submit statistical, narrative, and/or financial and program reports and summaries related to this contract as requested by NYSED.
  25. When subcontracting work under this contract, the vendor must use New York State registered Minority-Owned or Women-Owned Business Enterprises (M/WBE) for a
    minimum of 10% of its subcontracting needs. The subcontracting form must clearly define how and when this requirement will be achieved over the life of the contract and what vendors will be providing these services. For a listing of such providers, see: http://www.nylovesmwbe.ny.gov external link icon.

Test Administration Schedule

Develop and submit a detailed five year schedule/workplan describing how the vendor will develop and implement:

  • Two spring (March/April) field tests, 2011 and 2012.
  • Four spring (April/May) operational tests 2011, 2012, 2013 and 2014. Operational tests to be administered in 2011 and 2014 will consist of intact test forms that were administered during a prior school year and have been kept secure. The forms will be identified to the contractor after the contract is awarded.
  • One set of initial identification tests to be available for school use beginning by the start of the 2012-13 school year (September 2012). The vendor will assemble the tests using an operational test that has been administered in schools during a previous year. The vendor must also update the test-related materials needed to administer and score these initial identification tests, such as, teacher directions/manuals for teachers and administrators, non-scannable answer sheet template, rubrics, anchor/training papers, conversion charts, and answer keys.

Test Development

The vendor must adhere to the following parameters for the examinations to be developed:

  1. All tests must be aligned with the New York State ELA/ESL Learning Standards.
  2. Operational tests for 2011 and 2014 will consist of intact secure operational test forms that were administered during a prior school year. The forms will be identified to the contractor after the contract is awarded.
  3. Twenty-five percent of the items included in the 2012 and 2013 operational test forms will be from the previous year’s operational test form to ensure the stability of the vertical scale.
  4. For operational tests developed based on the current test specifications, the number and types of questions on each of the test forms must be the same as the current NYSESLAT. The NYSESLAT test samples are available at http://www.emsc.nysed.gov/osa/nyseslat/samplers.htm external link icon
  5. Following the adoption of revised Learning Standards, the vendor must develop a revised Test Blueprint. The Test Blueprint must specify the weighting of each standard across grade levels. (See Attachment A for template.)
  6. The revised test specifications must describe the types of questions and credit allotments for the various questions/parts of the test.
  7. It is expected that the Reading and Listening modalities will be comprised solely of multiple-choice questions. The Writing modality will include multiple-choice and open-ended questions and the Speaking modality will be comprised solely of open-ended questions.

Teacher Committees

The table below lists the teacher committees required for the NYSESLAT program and the approximate number of teachers required for each. Please see Attachment J for additional information.

Teacher Committees

Date

Year

Committee

Number of Teachers
April* 2010 Test Specifications Revisitation 45 - 60 NYS teachers
April 2010, 2011 Field Test Passage Review 25 NYS teachers
August 2010, 2011 Field Test Item Review 48 - 60 NYS teachers
July once, TBD Standard Setting based on Operational test data 100 NYS teachers
July 2011, 2012 Range Finding for field tests 60 - 90 NYS teachers
November 2010, 2011 Field Test final eyes review 14 NYS teachers
December 2011, 2012 Operational Test final eyes review 14 NYS teachers
* The Test Specifications Revisitation committee will be convened once during the duration of the contract at a data agreed upon by NYSED as soon as the revisions to the Learning Standards for English as a Second Language are approved by the Board of Regents.

All activities involving New York State teachers must be held within a radius of 20 miles of the NYSED building at 89 Washington Avenue in Albany, New York. The vendor is responsible for maintaining complete records of committee meetings, including sign-in and sign-out sheets, and providing these records to NYSED upon request.

Test Specifications Revisitation Committee

Test specifications revisitation committee meetings for each grade band and modality are to be conducted by the vendor to ensure alignment of the tests with the revised standards. This will occur as soon as practical after the revision of the learning standards for English as a Second Language are approved by the Board of Regents. The revised test specifications must be developed by the vendor in collaboration with committees of stakeholders, to include:

  1. Certified New York State (NYS) ESL and bilingual teachers, and a small number of NYS special education teachers.
  2. NYS school district administrators, including bilingual/ESL department chairpersons, principals, and superintendents.
  3. Members of the New York State Association for Bilingual Education.
  4. Members of the New York State Teachers of English to Speakers of Other Languages.
  5. Members of the New York State School Boards Association.
  6. Members of the New York State United Teachers.
  7. School assessment coordinators.

The resulting recommendations on revisions to the test specifications must specify the number and percent of items meeting each ESL learning standard and be broken down by item type. (See Attachment B for a sample of the current test specifications.)

The test specifications revisitations subcommittees are required to have representation from all geographic regions of New York State and must represent the diverse population of New York State. Three subcommittees must be convened for the following grade spans K-4, 5-8, and 9-12 and each subcommittee must be comprised of 20 members. A minimum of fifteen (15) of the members on each subcommittee should be certified NYS teachers.

The committee meetings must be held in the Albany area and the final recommendations for the Test Blueprint/Test Specifications agreed upon by the vendor and committee representatives must be submitted to NYSED for final review and approval. The committee members will meet for a total of three (3) full consecutive days.

The vendor is responsible for all the arrangements and costs for the test specifications revisitations committee and subcommittee meetings, including lodging, meals, travel, and honoraria at the approved New York State rates. The vendor is also responsible for arranging the meeting space and paying the rental fee. These costs must be budgeted for in the bidder’s cost proposal.

NYSED will identify appropriate and representative members whom the vendor will contact to determine availability and willingness to participate in the test specifications revisitations committee/subcommittees.  The vendor is not responsible for travel, meals or lodging of New York State employees associated with any event related to the contract awarded from this RFP.

Within one week after the committee meetings, the vendor is required to provide to NYSED a final comprehensive report describing the participants, the procedures followed to revisit and develop revised test specifications, the major decision points, and the final proposed test specifications. This report must be submitted to NYSED in an electronic format, and then also included in the annual Operational Test Technical Manual.

The test specifications revisitations shall take place once during the duration of the contract shortly after the revised ELA/ESL Learning Standards and core curricula are adopted by the Board of Regents. Following adoption of revised test specifications for NYSESLAT, the vendor must also develop and implement a plan to make substitutions of items to the initial identification tests so that they remain aligned to the revised NYSESLAT.

Content Review and Final Eyes Committees

The vendor will convene, maintain, and budget for the expenses for the following review committees:

  1. Content Review Committees (Passage Review and Item Review) must be convened to review test passages and items before field testing, to ensure their quality and fairness, and to advise NYSED to accept, reject or modify test items based on psychometrically substantiated test design and quality content, and to accept or reject passages. As part of this review, the committee shall examine each test passage and item to ensure compliance with standard 7.4 of the Standards for Educational and Psychological Testing. The committees do not select items but instead make recommendations to the vendor and NYSED on which items to include. Passage review committees should be divided into grade groupings (K-4, 5-8, 9-12) and item review committees must be divided into grade band subcommittees.
  2. Final Eyes Committees (Field Tests and Operational Tests) representing NYS geographically and demographically review the final proposed form of each field test and operational test before vendor printing. The first committee should review the K, 1-2, and 3-4 tests. A second committee should review the 5-6 and 7-8 tests, and a third committee should review the 9-12 tests. The same committees or similarly configured committees must review the initial identification tests assembled by the vendor from items selected from previously administered NYSESLAT operational tests. It is not necessary for the vendor to convene final eyes committees to review those operational tests (2011 and 2014) that will be comprised of secure intact operational test forms that were administered during prior school years.

Test Content

Test content must match the Test Blueprint/Test specifications (Attachments A & B) agreed upon by the vendor and NYSED. On each form and subtest, the raw score value representing English proficiency at each grade level must be no greater than 90 percent of the highest raw score value attainable. Each test must be constructed so that students do not have to earn a perfect or near perfect score on any subtest to demonstrate English proficiency. Bidders must propose designs for developing the first field tests administered under this contract. The test questions must be developed and demonstrably aligned with the test specifications (Attachment B), the ESL Learning Standards (http://www.emsc.nysed.gov/biling/resource/ESL/standards.html) and the ELA Learning Standards and Core Curriculum (http://www.emsc.nysed.gov/ciai/ela/elarg.html).

The vendor is responsible for writing all items that will be used in the field tests. NYSED prefers items written by individuals with a minimum of a Bachelors degree as well as a degree in the ELA/ESL content area, knowledge of appropriate questions for the target audience, experience in writing and editing items, and knowledge and training in the New York State Learning Standards. The vendor is permitted but not required to use New York State teachers to write items for the NYSESLAT if they meet the above minimum qualifications.

NYSED prefers that the vendor use passages commissioned expressly for this test. If the vendor proposes to use copyrighted authentic passages, the vendor must obtain approval of the copyright holder for use of the passage and adaptations to the passage on the printed test forms. These permissions must be obtained in writing prior to passages being used in field tests.

The vendor must:

  1. Develop items and field test items that will cover both extremes of the testing pool-easy and difficult.
  2. Develop at least ten multiple choice items per passage
  3. Write items that are clear and age/grade and proficiency level appropriate for each test.
  4. Ensure that the items and item types reflect the best educational research available at the time. Items must match the cognitive demands defined for each standard and be clear to the reader.
  5. Ensure the constructed-response items clearly define the expectations of the students, such as the necessity of providing details from a story or other piece of literature as support.
  6. Ensure multiple-choice questions are organized using an A B C D pattern. The use of "none of the above" or "all of the above" may not be used. Items having negative words such as "not" should be used only when necessary. There must be only one correct answer choice for each multiple-choice question.

Field Testing for 2011 and 2012 only

Field tests must be administered in Spring 2011 (March/April) for the operational tests to be administered in April-May 2012 and in Spring 2012 for the operational tests to be administered in April-May 2013. Currently, schools are asked to participate in field testing based on a multi-year field test schedule from NYSED (Attachment K), and they are generally cooperative in doing so. (There will be no field testing for NYSESLAT during the 2009-10 school year.) Participating schools do not receive a stipend. The following are the activities required around field testing:

  1. A detailed plan for the development and administration of the field tests must be provided to NYSED’s review and approval prior to field testing taking place in 2011.
  2. The field tests must be administered within six weeks of the administration of the operational tests.
  3. The vendor must administer the field tests separately from the operational tests. For security purposes, the field test materials must be noticeably different in appearance from the operational test materials.
  4. The vendor will select a statewide representative sample of NYS public, charter, and nonpublic schools to participate in field testing and notify the selected schools of the field tests. The samples must be selected in collaboration with NYSED and reflect the diversity of the State’s LEP/ELL population, including gender, ethnicity, and geographic locations.
  5. Field tests are administered by New York State public schools (including charter schools) and nonpublic schools that administered operational NYSESLAT to at least ten (10) students during the prior school year.
  6. The field tests must include the number of LEP/ELL students at each K-12 level that are representative of the NYS LEP/ELL student population. See chart below:
    LEP/ELL students

    Grade Band

    New York State LEP/ELL Student Population

    Target FT
    Sample N

    Actual FT
    Sample N

     

    Public N

    Nonpublic N

    State Total N

     

     

    K-1

    50110

    4816

    54926

    12000

    24000

    2-4

    57153

    4737

    61890

    12000

    24000

    5-6

    24600

    2380

    26980

    12000

    24000

    7-8

    22389

    1654

    24043

    12000

    24000

    9-12

    34583

    1255

    35838

    12000

    24000

    Total

    188835

    14842

    203677

    60000

    120000

    * Actual Sample N reflects the samples to be selected in order to achieve the target sample size.

  7. The vendor must collect a minimum of 1,500 usable field test answer papers per field test form. Since field test participation in NYS is not mandated, the vendor must over sample consistent with #6 and #10 in order to achieve the target of 1,500 answer papers written per field test form.
  8. The vendor must develop a method for soliciting schools to participate in field testing. The solicitation by the vendor for schools to participate in field tests must be explicit as to the grade levels and modality for which the school is asked to participate. Correspondence with schools must be approved by NYSED.
  9. At least two weeks prior to the established deadline for responding to field test solicitation, the vendor is required to send notification to schools selected for field test participation that have failed to submit their enrollments for participation.
  10. All schools that respond affirmatively to the field test enrollment must be provided by the vendor with the appropriate quantities of field test materials for that grade(s) and modality even if it appears to the vendor that more than the target number of answer papers will be achieved. Printing and shipping estimates are indicated in Attachment L. These estimates are built on a matrix that NYSED provides.
  11. The vendor will be responsible for the development, printing, and shipping of field test forms, teacher directions, student field test booklets, CDs with the Listening passages for grades 1 through 12, blank audio cassette tapes for the Speaking forms, and answer documents and directions for administering and returning the field tests to the vendor. See Attachment L for a detailed list of field test materials and quantities to be supplied to schools by the vendor.
  12. Field Tests are to be in mini-forms by modality such that students can generally complete the form in a 40-minute class period.
  13. The vendor must provide each participating school with an easy-to-follow set of instructions that explains how to administer a field test in the grade band and modality assigned to the school. Separate teacher directions must be developed for each modality (speaking, listening, reading, and writing).
  14. The vendor must design the answer sheets or scannable field test booklets for student responses to the field tests and submit to NYSED for review and approval.
  15. The vendor is required to propose a detailed plan to field test a number of items with at least 50% overage in number of items needed for each modality in each test for possible item replacements during item selection. The vendor should draw upon its experience to ensure that there will be enough quality items available to populate the operational tests. 
  16. The field test item pool is also required to be large enough so that the operational test forms needed can be developed using “successfully” field tested items, and so those items found unusable for an immediate operational test can be revised and reworked in order to be used in future field test cycles, thus building an item bank for NYSED’s use. The item bank may be requested by NYSED at any time during the contract period.
  17. The field test items and test materials must remain secure. The vendor must provide notice to schools on each page of the field test forms and administration directions that photocopying materials and any reference to test content is strictly prohibited. The notice must be as follows:
    Secure materials. All rights reserved. No part of this booklet may be reproduced or transmitted by any means. Use of these materials is expressly limited to the New York State Education Department.
  18. The vendor must track and collect all secure field test materials and inform NYSED of the results. The vendor is responsible for assuring that all field tests and related materials are returned in a timely and secure manner, and contacting schools, repeatedly if necessary, that do not return materials by the due date. The vendor must provide a list of schools to NYSED that have failed to return materials within 2 weeks after the due date and again after the vendor has made at least three concerted efforts to effect their return.
  19. All secure materials distributed to schools as part of the field tests must be serialized. The vendor’s records and the shipping manifests must indicate the range of serial numbers shipped to the schools. The vendor must also account for each individualized secure item in checking returned materials. Follow-up correspondence to each school concerning non-returned field test materials must include the serial number(s) of the missing materials.
  20. The vendor is responsible for paying the shipping costs for schools to return the field tests, audiocassettes, and CDs to the vendor.
  21. Range finding for Writing for scoring field tests must be conducted in Albany annually for the two successive years of field testing using certified NYS ESL, bilingual education, ELA, and bilingual special education teachers. A minimum of ten (10) to fifteen (15) members must comprise each grade band for range finding committees. Range finding committee members must represent different geographic and demographic populations of New York State.
  22. Range finding meetings will be arranged collaboratively between NYSED and the vendor. All range finding expenses will be paid by the vendor (excluding the expenses of NYSED staff that attend the sessions.)
  23. NYSED will provide to the vendor names of teachers to participate in rangefinding but the vendor is responsible for making all of the contacts to determine the teacher’s availability and willingness to participate and confirmations with such teachers.
  24. The vendor will develop models for scoring the open-ended questions and submit to NYSED for review and approval.
  25. The vendor must score the field tests, verify the results, forward the score results to NYSED, and produce a Field Test Technical Report for each of the two years of field testing. (See Attachment M for outline.)
  26. NYSED does not require that New York State teachers be used for the actual scoring of the field test items. However, scorers hired by the vendor for these field tests must have at least a bachelor’s degree and nine college credits in English and/or Education.

Operational Test Administration

  1. The vendor must prepare all test forms and all test-related materials to be printed, CDs with the Listening passages for each grade band for grades 1 through 12 (scripts for the Kindergarten Listening passages are to be included in the teacher directions), and DVDs to be used for training teachers in scoring the Speaking and Writing subtests. The operational scoring DVD must include information on the design and implementation of the NYSESLAT, use of the generic rubrics, and scoring the constructed-response items for Writing and Speaking. Instructions for more than one grade band may be on a DVD. Items from the operational test must be used. Videotaping of scoring DVDs must occur within 20 miles of the NYSED building to allow the participation of NYSED staff.
  2. The vendor must propose/construct the 2012 and 2013 operational test forms based on the field test results and research guidelines for item selection and additional field tested items for possible replacement.
  3. Forms construction meetings will be held at the NYSED building with NYSED and vendor staff to review the proposed forms and make necessary edits. The vendor must provide proposed forms and all field test statistical data for review. The vendor must provide NYSED staff with item statistics, including classical and Item Response Theory (IRT) statistics for the items selected for the operational test forms as well as the replacement items two weeks before the joint operational test form selection. At the time of form selection, the vendor must also provide NYSED staff with a mapping of all proposed items to the test specifications as well as the performance indicators associated with the possible replacement items.
  4. The final operational test forms will be jointly reviewed by the vendor and NYSED staff and signed off by NYSED.
  5. NYSED requires that New York State assessments be accessible to all students, including students with disabilities. Attachment E contains general recommendations regarding the development, presentation, and format of tests for visually impaired students. Guidance for schools regarding adapted tools and methods is also included. Bidders must ensure that tests developed meet the requirements described in Attachment E. The Guidance on Instructional Accommodations section at the end of Attachment E must be printed as a separate item and shipped by the vendor within the first eight months of the contract award to the school principals at public and nonpublic schools expected to administer the NYSESLAT.
  6. The vendor must provide full length Braille (see Attachment F for specifications) and large type editions (see Attachment G) of all test books for those schools that require them and special instructions, as necessary, for the administration of these forms. The reproduction by the vendor of the quantities specified in Attachment N of Braille operational test booklets and special instructions for their administration is a required component of this RFP.
  7. For Grades 1 through 12, the vendor must provide written transcripts of the Listening passages only to those schools that request them. These scripts are provided for administration, as an accommodation as specified in the student’s IEPs, only to hearing-impaired students who are not proficient in American Sign Language so that such students may read those passages. The vendor must also provide written instructions to schools on how they may request these scripts from the vendor.
  8. The vendor must provide separate test booklets for each of the subtests for every student taking the test. Grade level band, modality designations, and New York State English As A Second Language Achievement Test must appear on all test materials. Under no circumstances can a single test booklet contain all four subtests (Listening, Speaking, Reading, and Writing).
  9. Grade appropriate graphics, content, style, typeface, and font size must be proposed by the vendor and approved by NYSED.
  10. For those operational tests which consist of intact secure operational test forms that were administered during a prior school year, test forms will remain in color. However, NYSED has a preference for future NYSESLAT forms to be developed in black and white.
  11. All tests and test-related materials must be completed in the style and format prescribed and approved by NYSED. The vendor can access the style and format from the test samplers located on the NYSED web site. http://emsc.nysed.gov/osa/nyseslat/samplers.htm.
  12. An online web site and Teacher Evaluation Form must be developed and submitted to NYSED for review and approval. The web site must be in accordance with the New York State Web Based Information and Applications Policy found in Attachment O. Teachers will be directed to this web site in the Teacher’s Directions and School Administrator’s Manual for the test. The form will ask teachers to evaluate the test design and effectiveness of the NYSESLAT in measuring student achievement of English proficiency. It must allow for teacher comment and recommendations for improvement. The form must be completed by the vendor by the first week in February of each year of operational testing. The vendor must prepare quantitative and anecdotal summary reports on the results of the evaluation for NYSED by the end of July of each year.

Informational Materials for Schools and Parents

Near the start of the first school year in which operational NYSESLAT Tests based on revised test specifications are administered, the vendor must develop, print, and ship to schools informational publications. The publications must include a test sampler that includes the four modalities that describes the rubrics and the test design and informational brochures for parents. The test sampler may reuse sample questions included in previous test samplers provided the questions continue to be representative of the test format, given any changes made to the test format following the test specifications revisitations.

In addition to test samplers, the vendor must also provide informational brochures for parents, in user-friendly language. These must be developed, printed, and shipped by the vendor to the schools for dissemination to parents. The brochures must be made available by the vendor in English, Chinese (Simplified), Haitian Creole, Korean, Russian, and Spanish.

Two (2) copies each of the publications must be sent to each school and the district office, and a small volume (100 of each) sent to NYSED. Both publications must be prepared in PDF and html format for web posting by NYSED. Attachment N provides print quantities for these informational materials to be printed, duplicated, and shipped by the vendor.

Printing, Duplication, and Shipping of Operational Tests

For each of the four years of operational tests, the vendor is required to print the quantities of the NYSESLAT and NYSESLAT related materials, duplicate all DVDs and CDs, and ship all such materials to the schools requesting them. Materials to be printed and shipped include all operational test booklets, including those in Braille and large type, as requested by schools; technical manuals, teacher directions; school administrator manuals; printed copies of dictation for listening passages; scoring guides for local scoring that must include rubrics; annotated scoring exemplars; practice sets; and any other test-related materials necessary for valid test administration. Attachment N contains a detailed list of materials and quantities to be printed, duplicated, and shipped by the vendor. The vendor will be required to re-ship any missing or damaged materials.

The vendor must develop and send printed or scannable order forms to all schools, process operational test orders submitted by schools, send written confirmation of test orders via mail or e-mail to each school, input supplemental requests from schools, and develop and print manifests to pick and pack test materials ordered by schools. NYSED will provide the vendor with an Excel file annually in January that includes: school name, school address, school code, principal name, e-mail address, fax and phone number of the schools to which the vendor will send order forms. The building principal will be the vendor’s primary contact.

On the order form, the vendor must provide each school with the opportunity to specify an alternate school location to which the school may prefer to have its secure examination materials shipped.

All secure test materials for operational tests must be shrink-wrapped. Packages of secure materials that will be administered on different days must be printed in separate booklets and shrink-wrapped separately. The secure materials needed to administer Speaking sessions of the operational test must be packed and shipped to the schools separately from secure materials needed to administer the other three sessions: Listening, Reading, and Writing. NYSED will not entertain combining the shipping of materials for all four modalities.

The vendor must print and ship to the schools the quantities of Braille and large-type editions of the tests that are requested by schools and NYSED. The Braille editions must be accurately labeled on the front of each test with the grade band, subtest/modality, and the words “Braille Edition”. The large-type edition must be designed in a manner that recognizes the page-to-page break that provides for printing in booklet format, stapled in the spine by machine. The vendor must insure that the enlargement of graphics on large-type editions does not prevent students from getting the correct answer.

CDs for Listening must be labeled with color labels, specific to each grade level block test, and packed individually in either poly-boxes or clear slim packs.

The vendor must have defined procedures in place for order processing, test printing, distribution, and security. The vendor must ensure that it has the technical support and staffing levels necessary to provide an effective quality control program for all test materials.

The vendor must provide a customer service number which is staffed from 7:30 A.M.‑4:30 P.M. EST each business day to handle all ordering, printing, and shipping-related questions immediately after the vendor’s notification letter to schools of ordering procedures, and prior to and during the test administration period.

The vendor is NOT responsible for the solicitation of orders, printing, or distribution of the initial identification tests or related materials (teacher directions/manuals for teachers and administrators, non-scannable answer sheet template, rubrics, anchor/training papers, conversion charts, and answer keys) assembled by the vendor. NYSED will be responsible for these activities. However, the vendor must provide NYSED with the print ready files for all such materials and for reformatting/revising into large-type and Braille.

Return of Operational Tests to the Vendor

  1. The vendor must budget and arrange for each school to return the operational tests to the vendor and to confirm receipt using pre-paid shipping labels. The vendor must also contact, repeatedly if necessary, all schools that do not return the operational test materials.
  2. NYSED requires that the vendor provide a weekly report to NYSED on the status of operational test returns and make its best effort to attain a 100% return of materials.
  3. The vendor must store all operational test student answer documents that are returned by the schools for one year after the administration of the test. These documents must be stored in a retrievable fashion. After one year, student responses may be securely destroyed in a manner that protects students’ privacy.

Scoring of Operational Tests by Local Schools/Vendor

  1. The vendor must provide a customer service number for test administration and scoring questions from local schools concerning the administration of the tests and the scoring of the constructed response questions. Schools typically inquire about exam scoring issues, the answers to which must be limited to scoring procedures, or about untypical responses to questions. The customer service number must be adequately staffed from 7:30 A.M.–4:30 P.M. EST each business day beginning the first day of testing and for four weeks after the testing period to handle all test-related questions. Wait time for the customer service number should not exceed three (3) minutes.
  2. Multiple-choice items are not scored locally. Constructed-response items are scored locally by teachers in the public schools and participating non-public schools. Scores for the constructed-response items are recorded by the teacher on an answer sheet provided to the school by one of the State’s scanning centers. There are approximately 17 different local or regional scanning centers responsible for providing answer sheets to New York State schools.
  3. After the local scoring period, answer sheets are scanned at the same scanning center which provided them to the school. The vendor must coordinate with NYSED and the scanning centers to develop a file layout for the resulting scan records.
  4. After scanning their answer sheets, the scanning centers send their data files to a central repository – the State’s Student Information Repository System (SIRS) – which merges all files into one raw data file and sends this to the vendor.
  5. The vendor will:
    1. Receive the raw data file for all test takers from SIRS.
    2. Score each the multiple choice components for each student, sum the raw scores for each modality, and determine the student’s scale scores and performance level.
    3. Insert scaled scores and performance levels to the data file.
    4. Send the scored file back to the Student Information Repository System.
  6. The vendor must send individual student score reports either in print or on CDs or DVDs, by roster, to the schools by July 30, 2011. For the three subsequent years, the student score reports must be sent to schools by July 20. The format for the individual student score reports must be proposed by the vendor and approved by NYSED. The individual student score reports must include, at a minimum, the student’s raw scores for all four modalities, scale scores and performance level, combined as follows, Listening & Speaking and Reading & Writing, and overall performance level. The score report must also include information for parents and teachers on how to interpret the student results. The generic guide for interpreting the results must be translated into five languages (Chinese-Simplified, Haitian Creole, Korean, Russian, and Spanish).
  7. For each year of operational testing, the vendor must produce state level reports that include, at a minimum:
    1. The total number of identified LEP/ELL students statewide;
    2. The number of students assessed at each grade level by school, school district and statewide;
    3. The percent of students scoring at each achievement level for each assessment component and for the total assessment;
    4. Achievement level descriptors,
    5. Percentage of students who responded correctly to each multiple-choice test question;
    6. The mean raw score for each open-ended test question;
    7. The standard and performance indicator from the core curriculum associated with each test question.

These reports must be provided by the vendor electronically via the secure FTP site to NYSED by July 31, 2011 and by July 20 for the subsequent years.

The same elements must also be contained in the technical report that is required to be submitted to NYSED for each operational test. For reference of what NYSED’s expectations are regarding a technical report see Attachment I.

Training for Local Scoring of Operational Tests

For the first administration of new NYSESLAT operational tests that are developed in accordance with the revised test specifications, the vendor must provide training for local scoring. The vendor must provide twelve (12) turnkey training sessions for teachers and administrators that train teachers in how to score the open ended questions in the Writing and Speaking subtests.  Emphasis in the training should be on those elements that have been revised. The training must include information on the design and implementation of the NYSESLAT, use of the generic rubrics, and scoring the constructed-response items for Writing and Speaking. Items from the test samplers, not operational items, should be used in training.

The vendor will be responsible for developing training materials, arranging for training sites, and any and all associated costs of training. Teachers attending the training do not receive honoraria or reimbursement for travel or lodging costs. School districts are not reimbursed for substitute teachers.

Sites for the turnkey training must be geographically distributed across the State, typically in: Nassau, Suffolk, Southern Westchester, Mid-Hudson, Capital Region, North Country, Syracuse, Rochester, and at least four in New York City. Up to 100 teachers may attend each training session.

The vendor must coordinate the turnkey training activities with NYSED, the Bilingual ESL Technical Assistance Centers (BETACs), the local BOCES, and the Big Five city school districts for the first administration of operational NYSESLAT tests developed based on the revised test specifications.

The training sessions must be completed by the last week in March. Specific dates and locations will be determined in collaboration with NYSED, BETACs, BOCES, and Big Five city school districts, which may be contacted to assist with the planning of the training.

The vendor must provide to NYSED a technical report on the training for scoring. The report must contain a summary of the entire turnkey training process including the following:

  1. List of participants, titles, school districts and ethnicity
  2. List of locations
  3. Evaluation Form results
  4. Copy of all training materials

Re-scoring of Operational Tests

  1. The vendor must select a representative sample of 10% of the operational Writing test papers for each grade band to be re-scored. The sample for this audit must be selected in cooperation with NYSED.
  2. The Student Information Repository System will provide electronic data files to the vendor of the local scores for open-ended questions for each school as part of the larger data set sent to the vendor.
  3. The vendor must provide scoring training to the re-scorers they hire using the same scoring training materials used by schools to score the operational tests. Scorers hired by the vendor for these operational tests must have at least a bachelor’s degree and nine college credits in English and/or Education.
  4. The re-scoring of the operational examinations must be completed by the first week in September.
  5. The vendor must provide NYSED with data files on the original scores and re-scores and an electronic report on the re-scoring. The report must then be included in the annual operational test technical manual. (See Attachment P for the outline.)

Research

  1. The vendor must conduct scaling, equating, and item calibration; provide analysis of test and item psychometric properties.
  2. The vendor must develop technical manuals/reports for each operational test, each field test, the initial identification test, and re-scoring audit, after each test administration, based on the technical manual outline provided by NYSED (Attachment P).
  3. The vendor must provide technical reports to the Research Division of the Office of State Assessment for the test specifications revisitation and for the standard setting.
  4. The vendor must provide field and operational test statistical data about each test question after each administration so that NYSED may conduct subsequent secondary analysis at its discretion.

Analysis of Field Test (FT) Data

The vendor must provide statistical analyses for each FT administration. The analyses must include, but not be limited to: FT data analysis, classical item analysis, form reliability analysis, IRT statistics, FT equating, and differential item functioning (DIF).

  1. FT data analysis: File merging, data clean up, and the evaluation of sample representativeness.
  2. Classical item analysis: N-count, item mean, item-total correlations (point-biserial), percent choosing each answer choice for multiple-choice items, percent receiving each possible score point for open-ended items, and distracter analysis.
  3. Form reliability analysis: Form reliability and the inter-rater reliability analysis for the constructed-response items.
  4. IRT statistics: The field test data must be calibrated using item response theory (IRT) model, including IRT calibrations (item difficulty, discrimination) and item fit evaluation.
  5. DIF analysis: The Mantel-Haenszel process for multiple-choice items, a derivative process for open-ended items, or an IRT-derived process for both item types must be used to evaluate DIF on field test items for gender, ethnicity, NRC, and for population focal groups, identified by NYSED. The minimum sample size for a focal group is 500.

A research file must be prepared for NYSED containing student item level data as well as the demographic information necessary to replicate all analysis. If the vendor is using proprietary software to scale and equate the test forms, the vendor must provide NYSED’s research staff with access to that software at no additional charge to NYSED for the duration of the contract and three years after the end of the contract and must provide free training in its use.

Equating

The vendor must use an IRT model to calibrate, equate, and scale the NYSESLAT.  Currently, the Rasch and Partial Credit Models are used for NYSESLAT.

Four scales must be developed and equated from form to form for each year: (1) Listening and Speaking, (2) Reading, and Writing, (3) Comprehension (from Reading and Listening), and (4) Total Score (for accountability purposes only)

The equating must provide for year-to-year comparisons for each test through the development of the four common scales.

Bidders must propose the most appropriate IRT equating designs for both field tests and operational test equating.

For quality control purposes, two simultaneous equatings must be performed and the equating results must agree.

Field test equating: The operational test will serve as common anchor items to allow field test items to be calibrated on the operational scale. The common scaling of field test items to the operational scale must be based on representative statewide samples.

Operational test equating: operational data must be used for item calibrations and equating. The new operational tests will be equated to the baseline year using the operational item parameters.

The bidder must propose how to link the new tests with the prior year’s tests, in order to provide continuity. The vendor must also propose how to link the Initial Identification Test to the NYSESLAT. Currently, an IRT-internal common-item design is used for linking the NYSESLAT forms across years. Each year, approximately 25% of the test questions from the previous years’ test are included in the current year’s test and must continue to be included in each year’s test.

The alignment of test characteristic curves and standard error curves of the baseline operational tests and the new operational forms (e.g., 2011, 2012) must be evaluated.

Vertical Scaling

Presently, the NYSESLAT consists of five forms, delineated by grade level bands.  Following the test specification revisitation, the NYSESLAT will consist of six forms, delineated by grade bands. A common vertical scale across grade level must be developed to evaluate the growth of individual students from year to year for each of the four scales (Listening & Speaking, Reading & Writing, Comprehension, and Total). Cut scores must be determined for every grade level from K‑12.

Test scores for each grade level band must be reported on a unique scale. The scale’s range must be discussed with NYSED in view of vertical scale results.

The scales must each be demarcated by four performance levels, determined by formal standard setting studies. The performance levels are: Beginning, Intermediate, Advanced, and Proficient.

The vendor must establish the linking between the new operational vertical scales and the prior year’s vertical scales in order to provide continuity.

Pre-and Post-Measurement Review, and Standard Setting

The Standard Setting process for the revised NYSESLAT consists of three activities: the Pre-Measurement Review Meeting, a traditional Standard Setting using the item mapping method, and the Post-Measurement Review Meeting. The main purpose of these meetings is to obtain cut score recommendations for the Department for each grade level and modality for the revised NYSESLAT.

The vendor will be expected to conduct Pre-and Post-Measurement review meetings in Albany about one month before and immediately after Standard setting for the revised NYSESLAT operational tests. About one to two months after the administration of the operational tests, a three day Standard Setting is to be conducted by the vendor in Albany.  The meeting will include approximately 100 teachers, who will be divided into groups of 25 teachers in each of four rooms.

The teachers in the first room will be making recommendations on the cut scores for the NYSESLAT tests administered in Grades K, I, and 2; teachers in the second room will address Grades 3, 4, 5,and 6; teachers in the third room will address Grades 7-8, and teachers in the fourth room will address grades 9-12.  Each room will need at least five tables to accommodate the participants, vendor and NYSED staff.  NYSED will provide the names and contact information of the teachers that the vendor will contact to determine availability and willingness to participate.  Each of the five tables will have a table leader selected by NYSED. The leader’s role is to facilitate table discussion, ensure participation of all table members and report out summaries of table discussions to the larger group in each room.

In the afternoon of the third day of Standard setting, a Synthesis Group composed of all the table leaders for all three rooms will meet.  Their purpose is to review the recommendations on cut scores arrived at for each grade level in the three separate rooms and to make recommendations on revisions to those determinations that help to provide a logical vertical articulation across the K-12 spectrum.

The next day the vendor is to conduct the Post-Measurement review meeting with the same attendees who attended the Pre-Measurement Review Meeting.  The purpose of this meeting is to make recommendations on cut scores to the Department based in a review and integration of the recommendations that had been provided at the Pre-Measurement review Meeting as well as at the Standard Setting.

The vendor is responsible for:

  1. arranging for the hotel conference room space in Albany where these meetings will take place and where teacher and administrators participants residing more than 30 miles from Albany will be given lodging.
  2. contacting and sending formal invitations to the individuals whose names NYSED has provided to the vendor by NYSED.
  3. securing signature of participants on the Non-Disclosure agreements provided to the vendor by NYSED.
  4. paying for hotel space, meals, travel reimbursement for the invitees.  In addition, the vendor is to provide an honorarium to the participants in the Standard setting but not to the participants in the Pre-and Post Measurement Review Meetings.
  5. agenda development, handouts, Power Point presentation, sign in and out sheets for participants for the two meetings.
  6. maintaining the security of materials and arranging for secure destruct services at the conclusion of Standard Setting.
  7. submission to NYSED of the Standard setting Technical Report which includes sections on the Pre-and Post- Measurement review Meetings.

A comprehensive final report on the standard setting study is required. (See Attachment Q for outline.) A draft of the technical report must be submitted to NYSED within two weeks of the completion of the standard setting. The final technical report must be submitted within two weeks of receiving requested changes from NYSED.

Required Technical Reports

The vendor must provide a comprehensive technical report for each field test and operational test administration. (See Attachment M and Attachment P respectively for an outline of these reports.) A separate technical report for the standard setting must also be provided. (Attachment Q provides an outline for the standard setting report.)

Prior to submission of draft reports to NYSED, all reports must be carefully proofread and edited. The draft technical reports must then be reviewed and approved by NYSED. After approval, hard and electronic copies of the final technical reports must be provided to NYSED in the quantities and format specified in Attachment R. All reports must be provided to the Office of State Assessment research unit.

References and Qualifications

The bidder must submit documentation regarding experience in conducting the activities required in this RFP which include the development of large-scale assessment programs, including any experience in developing English as a Second Language and language acquisition tests, and the printing and dissemination of all materials for large-scale assessment programs. Bidders must document experience in test development of open-ended questions using rubrics, including field testing, for large-scale assessment programs. Qualifications for test construction, administration and research, as detailed in the RFP, must be documented.

A minimum of three (3) current professional references to substantiate qualifications is required. The references must include the name, title, company name, address, email address and phone number. Do no use NYSED staff as references. NYSED will contact references to check a bidder’s qualifications. The bidders must ensure that the references are current and information submitted for references is accurate. Two points will be subtracted from the bidder’s score for incorrect or outdated reference information which results in preventing NYSED from making contact with a reference.

Project Management

The vendor must submit thorough documentation demonstrating personnel experience and expertise congruent with staff assignments for all aspects of the tasks outlined in the RFP. The vendor shall provide plans and procedures for ensuring that staffing is commensurate with the project’s scope of work throughout the duration of the contract and immediately notify NYSED of any changes in staffing, including any changes in subcontractors.

Sufficient information should be presented in the proposal to NYSED detailing the efficacy of the proposed staffing solutions for this project and organizational arrangements of the same.  This information must include:

  1. An organizational diagram that indicates staffing arrangements for this project.
  2. The time commitments of the proposed staff. Time commitments must be described in terms of projected hours per week devoted exclusively to the NYSED project.
  3. A detailed description of responsibilities for all proposed staff.
  4. Current detailed resumes for all staff members associated with this project.

Program Manager

The bidder must identify a program manager to be the central point of contact with NYSED for this contract. The program manager and all other key personnel of the vendor involved with this contract must be identified by name and experience in writing to NYSED and approved by NYSED. The program manager must not have primary responsibility for any other contract with NYSED or for a large scale testing program for another state.

The requirements for the program manager are as follows:

  1. Successful completion of a Bachelor’s Degree from an accredited college or university.  Master’s Degree or beyond is preferred.
  2. Project management certification through the Project Management Institute (PMI) as a Project Management Professional (PMP), or other recognized project management certification is preferred. If applicable, the project management certification must be submitted with the bid.
  3. Previous experience in a K-12 public education project.
  4. Demonstrated knowledge of testing procedures and strong organizational, managerial, and communication skills.
  5. Knowledge of New York State’s assessment program is preferred.
  6. Experience working with large scale LEP/ELL test programs is preferred.
  7. At least three years of experience managing large assessment projects from conception through completion following industry recognized project management methodology preferred.

Program Manager Responsibilities

  1. Submit all deliverables and other work products in the manner designed by NYSED for review and approval prior to any release or distribution by the vendor. All materials must be proofed for errors prior to submission to NYSED. The program manager must submit materials to NYSED according to a mutually agreed upon timeline to allow for appropriate review.
  2. Develop and submit an annual detailed project plan/schedule to NYSED for review and sign-off.  The purpose of the plan is to provide NYSED with an overall analysis of the methods the vendor will utilize to perform all aspects of the contract in the required timeframe. The vendor will not perform work on the project until NYSED has accepted the vendor’s annual project plan.
  3. Maintain accurate, up-to-date information of the current status of all vendor and subcontractor(s) work on the project, and communicate such to NYSED in a timely manner.
  4. Submit monthly status reports to NYSED that give an update on all phases of the development and implementation of the project and progress made in accordance with the detailed project plans and contractual requirements.  Such monthly reports must include the following:
    • A current detailed listing of all work and activities completed and in progress with corresponding dates
    • A detailed list of upcoming work and anticipated completion dates
    • A list of all information needed from NYSED in order to proceed with work
    • External dependencies, if any, that may affect the schedule, and the variance by which it would affect the schedule
    • Dates that correspond to all timelines and schedules for deliverables
  5. Travel to Albany, NY to meet with NYSED program staff.  A one-day start-up meeting is required to be held in Albany at the beginning of each contract year. The first meeting must be held no more than one month after final approval of the contracts.
  6. Coordinate and participate in quarterly status meetings that are one full day and monthly conference calls to discuss the status report and any issues related thereto. At least every other quarterly status meeting must be conducted face-to-face in Albany, New York. Provide minutes of all meetings to NYSED for review and approval. These quarterly meetings can be scheduled to coincide with other activities being held in Albany.
  7. Oversee the development and implementation of changes as necessary to ensure that the projects remain within specified scope and are within time, cost, and quality objectives.
  8. Appropriately and effectively communicate with teachers and NYSED staff. This includes the effective and professional facilitation of trainings, technical assistance, and relationship building with all involved stakeholders.

Notification Procedures

In the event that a problem or potential problem arises with regard to the quality, timeliness, or any other issue with respect to deliverables and services at any time during the contract term, regardless of when the problem arises, the program manager must immediately notify the Director of State Assessment, or his/her designee, via telephone and in writing of the issue and the vendor’s proposed solution and shall also include this issue and the NYSED approved solution in any subsequent reports.

NYSED Approval

NYSED shall have approval authority over all aspects of the NYSESLAT program including, but not limited to, the following:

  1. All material, products, and services produced by the vendor. This includes, but is not limited to, all test forms (test booklets and answer documents), brochures, guides, student information labels and answer sheet identifiers, manuals, and any and all other printed or audiovisual materials produced for this project. In addition, all test forms produced in Braille or large type must be reviewed and approved by the NYSED before printing and dissemination. 
  2. All development plans and timelines for item development, field-testing items, item replenishment, scoring, and reporting.
  3. All schedules, including but not limited to training schedules, rollout schedules, implementation schedules, scoring/reporting schedules, and item review schedules.
  4. All reports, including but not limited to technical reports (ongoing and post-operational), score reports, and field test reports.
  5. All new and revised assessment items developed by the vendor to ensure alignment to current and revised content specifications.
  6. Any changes to staff assigned to the project, including the program manager, on and after award of the contract and commencement of work for the duration of the contract.
  7. Any change orders. Any changes to services, deliverables, or timelines under this procurement must be in accordance with a written description, clearly justifying the need for the change and identifying all rational for the change. The New York State Office of the State Comptroller is the only entity that has the authority to approve the modification of an agreement between NYSED and a vendor. No work that is not already contracted for should be started by the vendor prior to its official approved by NYSED and the Office of the State Comptroller.

Monitoring and Evaluation

NYSED will monitor and evaluate the progress of the vendor to meet the contractual requirements through monthly conference calls and quarterly status meetings coordinated by the program manager. The evaluation will emphasize the bidder’s ability to meet timelines and supply deliverables at or under budget.

Ownership

  1. All examinations (including field and operational test items in both print and computer file format) and examination-related materials are the exclusive property of NYSED and cannot be used by the vendor or any subcontractor for any purpose other than what is defined in the contract.
  2. NYSED is the sole owner of outputs resulting from the work proposed in the RFP, including, but not limited to, item banks, technical documents, test questions developed for field tests, operational tests, all scoring materials, test samplers, manuals, scoring rubrics, score scales and all guides, booklets, papers, and reports generated under this contract.  The vendor must be prepared to deliver all or part of these outputs to NYSED at any point during the term of the contract.  These materials can only be used by NYSED.
  3. All such outputs are to be turned over to the State in print/or electronic form, as specified by NYSED, prior to the final payment to be issued under this agreement. The State may accordingly, at its discretion, make public, and may publish on its website, all examinations and related materials noted in the preceding paragraph after administration of the operational examination. The vendor may use such publicly disclosed materials in the same manner as any other party in accordance with the terms posted on NYSED’s website.
  4. All materials are to be delivered, as appropriate, in copyright form.  Where original works are included in the body of the material, an acknowledgment statement must be included, setting forth the copyright information with respect to such original works.

Terms and Conditions

  1. All tests and test-related materials submitted by the contractor to and/or reproduced for NYSED pursuant to the proposal are required to be materials that are not already accessible to New York State teachers or students.
  2. All materials written or revised are to be held strictly confidential and must not be copied, duplicated, disseminated, or discussed, unless authorized by NYSED.
  3. All proposal deliverables must be submitted within the timelines specified by NYSED in the RFP.  (See the Schedule of Deliverables in the Submission section of this RFP.)
  4. Payment to the vendor will be based upon invoices that may be submitted to NYSED after the activity is completed and/or deliverables have been submitted to NYSED. Payments will be made based upon receipt of deliverables that have been approved by NYSED.
  5. If Contractor also develops or publishes textbooks or practice tests for purchase by public school districts, charter or nonpublic schools or individuals within the State of New York, Contractor agrees that, when promoting to New York State customers such textbooks, ancillary materials, and/or practice tests, Contractor will not make any reference to Contractor having been awarded the contract pursuant to this RFP or Contractor's performance of the services for NYSED contemplated by such contract.
  6. If the Contractor is under ownership with another entity that develops or publishes textbooks or practice tests for adoption within the State of New York: a) ("Related Publisher") the restriction against disclosure  as stated in 7 above applies to such Related Publisher; b) the Related Publisher is restricted from providing any information to Contractor pertaining in any way to content uses, or proposed to be used, in the Related Publisher's development of textbooks or practice tests for adoption within the State of New York; and c) the Related Publisher is restricted, when promoting to customers within New York State, the Related Publisher's practice tests and/or textbooks developed for adoption within the State of New York, from making any reference to the common ownership of the Related Publisher and Contractor or any reference to Contractor's performance.

Considerations and Other Issues

Upon completion or termination of the contract awarded as a result of this RFP, the vendor will use its best efforts to assist NYSED in completing a seamless transition to any successive vendor and/or NYSED. This shall include, but not be limited to, assisting NYSED in developing and implementing a feasible transition plan in advance of the anticipated expiration, cancellation, or termination of the contract.

The vendor agrees to cooperate fully with any successive vendor and NYSED, and refrain from any activity that would interfere with the successful implementation of a seamless transition. The vendor shall provide at no cost to NYSED (a) all items, reports, materials, data, and equipment owned by NYSED in the vendor’s possession, and (b) any information reasonably useful to and requested by NYSED in developing a request for proposal for a successive vendor, prior to the expiration, cancellation, or termination of the contract.

The vendor will work with NYSED to determine the optimum format of providing data and information that will optimize the reuse of this data and information by any successive vendor. The vendor will also provide NYSED with a list of all computer programs and software tools necessary to allow an end user to read and export any data provided by the vendor under this contract.

Contract Period

One contract resulting from this RFP will be awarded for a term of five (5) years. The contract will begin on April 1, 2010 and end on March 31, 2015.

At the end of any contract term otherwise provided for herein, if a replacement contract has not yet been approved in accordance with State law, any contract awarded hereunder may be extended unilaterally by the State, upon notice to the contractor, at the same terms and conditions, including all contract pricing, for a period of one month, if funding is available and/or services are needed.  Additionally, this extension may be for a period of up to three months with the concurrence of the contractor.  However, any extension will terminate immediately upon approval of the replacement contract except where a period for transition of contractors has been previously provided for.

Subcontracting

  1. Subcontracting will be limited to forty (40%) of the annual contract budget.
  2. Subcontracting is defined as non-employee direct personal services and related incidental expenses, including travel.
  3. All subcontractors are paid directly by the contract holder.  In addition, all payments to subcontractors must be listed separately on invoices submitted to NYSED for payment.  This includes name of subcontractor, dates of services, activities and amount.
  4. Subcontracting activities, vendor names, and costs must be specifically indicated in the Subcontracting Cost Form located in the Submission Documents. This form must be updated on an annual basis or every time a subcontractor is added or deleted.  The updated form must be submitted to NYSED. NYSED reserves the right to approve all subcontractors.
  5. When subcontracting work under this contract, the contractor must use New York State registered Minority/Women-Owned Business Enterprises for a minimum of 10% of their subcontracting needs. The subcontracting form must clearly define how and when this requirement will be achieved over the life of the contract and what vendors will be providing these services. For a listing of such providers, see: http://www.nylovesmwbe.ny.gov external link icon.
  6. If the subcontractors change during the contract term, NYSED must be notified and reserves the right to reject any subcontractor considered by the vendor.  A list of subcontractors hired by the vendor and the amounts of those contracts must be included in the proposal on the Subcontracting Form located in 5.) Submission Documents.

New York State Education Department’s Minority/ Women-Owned Business Enterprise Procurements Policy

It is the policy of the New York State Education Department to provide certified minority or women-owned business enterprises and small business enterprises with equal opportunity to Department procurements. Article 15-A of the NYS Executive Law, and Section 163 (6) of the NYS Finance Law were enacted to promote fairness in, and greater access to, the procurement process. The Board of Regents and the Commissioner are committed to a workforce that is representative of the many communities we serve, and to business practices which likewise reflect this commitment. Integral to the achievement of these goals is the increased utilization of women and minority-owned enterprises (M/WBEs) as vendors of the goods and services purchased/contracted by the Department. In doing so, we will further support our mission "To raise the knowledge, skill and opportunity of all the people in New York."

Every Department employee involved in the procurement process is responsible for ensuring both compliance with the above referenced laws and the maximum possible use of M/WBEs consistent with procedures developed jointly by the Office of Fiscal Management and the Office for Diversity, Ethics and Access.

Back to top

Submission

Documents to be submitted with this proposal

This section details the submission document or documents that are expected to be transmitted by the respondent to the State Education Department in response to this RFP. New York State Education Department shall own all materials, processes, and products (software, code, documentation and other written materials) developed under this contract. Materials prepared under this contract shall be in a form that will be ready for copyright in the name of the New York State Education Department. Any sub-contractor is also bound by these terms. The submission will become the basis on which NYSED will judge the respondent’s ability to perform the required services as laid out in the RFP. This will be followed by various terms and conditions that reflect the specific needs of this project.

Project Submission: The proposal submitted in response to this RFP must contain eight (8) copies of the Technical Proposal (one bearing an original signature), three (3) copies of the Cost Proposal (one bearing an original signature), and one (1) electronic version in Microsoft Word (CD format) of both the technical and cost proposal. The Cost Proposal must be returned in a separate envelope. All certifications and assurances must be returned with the Technical Proposal. The proposal must be received by December 21, 2009 at 3:00 PM in Albany, New York.

Any proprietary materials considered confidential by the vendor must specifically be so identified, and the basis for such confidentiality must be specifically set forth in the proposal.

The proposal should be prepared simply and economically, avoiding the use of elaborate promotional materials beyond those sufficient to provide complete presentation. If supplemental materials are a necessary part of the technical proposal, the bidder should refer to these materials in the technical proposal, identifying the document(s) and citing the appropriate section and page(s) to be reviewed.

Bidders should specify all details and dates required to evaluate the technical proposal and should not include aspects of the project plan which are to be determined only after the award of a contract. Optional deliverables to be provided only at an additional cost should not be detailed and will not be considered.  Contractual terms, conditions, and assumptions should not be included in the proposal.

Technical Proposal (70 Points)

Your Technical Proposal should include the following:

  1. Response Sheet for Bids
  2. All Forms and Assurances located in Submission Documents including original signatures where necessary
  3. Mandatory Requirements Certification, Signature Required
  4. Project Description, organized in a binder and clearly labeled with tabs naming each section, in the order outlined in the following section, Criteria for Evaluating Bids.

Cost Proposal (30 Points)

The original plus two copies of the completed Cost Proposal must be mailed in a separate envelope labeled RFP #09-021 Cost Proposal Do Not Open and must include the following:

  1. Bid Form Cost Proposal-Schedule of Deliverables
  2. Five Year Budget Summary, Signature Required
  3. Subcontracting Form

Back to top

Evaluation Criteria and Method of Award

This section begins with the criteria the agency will use to evaluate bids, and closes with the "method of award" or how the contractor will be selected. This will be followed by various terms and conditions that reflect the specific needs of this project as well as New York State contract guidelines and requirements.

Criteria for Evaluating Bids

All complete proposals received by the deadline will be reviewed using the following criteria and ratings. Applicants must ensure that all components of this application request have been addressed, the required numbers of copies have been provided, all forms and assurances have been completed, and the original signatures in blue ink are included as required.

An evaluation committee of NYSED staff will complete a review of all proposals submitted that meet the requirements outlined in this RFP. The committee will review each proposal based upon the submitted proposal and the requirements of the RFP only. Bidders should not assume that committee review members will be familiar with the current program or have any previous experience with the bidder. Appropriate description should be included to inform review committee members about the bidder’s qualifications and capacity to perform all required deliverables. The review committee will determine compliance with the mandatory requirements described in the RFP and retains the right to determine whether any deviation from the requirements of the RFP is substantial in nature and may reject any and all proposals that do not meet these requirements.

Proposals receiving at least 42 of the 70 points available for the technical proposal will move to the next step of the process. Proposals with a score of less than 42 points for the technical proposal section will be eliminated from further consideration.

Technical Criteria (70 Points)

The Technical Components should be organized in a binder and clearly labeled with tabs by individual sections (i.e. A-1, A-2) in the following order:

Section A: Test Development (10 points total)
The proposal must describe in detail how the bidder will:
Section A-1: Develop multiple-choice and constructed-response items.
Section A-2: Develop the field tests, including how the bidder will match field test results to operational test results by student.
Section A-3: Develop an administrative handbook that includes information regarding administering the operational tests.
Section A-4: Develop scoring rubrics for constructed-response items and scoring training materials for field tests, including annotated practice and anchor sets.
Section A-5: Conduct teacher committee meetings for each test level, including a description of how the bidder will ensure that the committees will provide a quality of products.

Section B: Test Administration (10 points total)
The proposal must describe in detail how the bidder will:
Section B-1: Develop test booklets, including Braille and large-type editions; technical manuals for teachers and administrators; printed copies of dictation; and CDs for Listening.
Section B-2: Assure that the bidder has the technical support and staffing capacity to provide effective quality control for all test materials.
Section B-3: Develop training materials for local operational test scoring, including scoring guides, DVD production, and duplication and distribution to public and nonpublic schools.
Section B-4: Provide twelve (12) turnkey training sessions for teachers and administrators on local scoring following revision of the test specifications.
Section B-5: Conduct range finding for scoring.
Section B-6: Re-score a 10% sample of each operational test band in the Writing modality, in order to evaluate the scoring of the NYSESLAT.

Section C: Quality of Appropriate Security Measures (10 points)
The proposal must describe in detail how the bidder will:
Section C-1: Provide secure facilities, documented procedures, storage and retrieval, surveillance, printing and shipping of field tests and related test materials, and transmission of data files. Security measures must, at a minimum, be consistent with Attachment H.

Section D: Research (5 points)
The proposal must describe in detail how the bidder will:
Section D-1: Validate the appropriateness of the field test sample for each grade level and modality.
Section D-2: Provide statistical analysis of the field test administration, as described in the RFP, including a research file.
Section D-3: Conduct standard-setting studies for each of the grade level bands for the four performance levels, including a description of the standard setting process.
Section D-4: Propose an appropriate equating design and equating tests from form to form in all modalities to provide for year-to-year comparisons.
Section D-5: Develop of scales for each grade level band, delineated by the four performance levels.

Section E: Appropriateness of Workplans (10 points)
The proposal must provide:
Section E-1: A feasible plan for the test construction, field testing, administration, scoring, re-scoring, and data analysis described in the RFP, including specifics on major activities, all deliverables, and timelines.

Section F: Printing and Shipping of Operational Tests (20 points)
The proposal must describe in detail how the bidder will:
Section F-1: Producing printed tests and test related materials including audiotapes and CDs that are high quality, error free, and meet NYSED’s exacting standards for format, clarity of identification, accessibility, and security.
Section F-2: Processing of school test orders and pick and pack of ordered materials.
Section F-3: Shipping tests and related test materials, including CDs, in a timely and secure manner using a carrier with a tracking system.
Section F-4: Documentation of experience in printing and shipping large-scale examination programs statewide, including the name(s) and addresses of the institutions for which such experience was accrued and the dates of such experience.

Section G: Quality of References and Qualifications of Project Staff (5 points)
Section G-1: The proposal must identify a program manager and key staff including names, an organizational diagram, time commitments of staff, resumes, qualifications, and previous experience.  A copy of the project management certification, if applicable, of the manager assigned to this project must be included in the proposal. The job duties of these persons must be clearly described. A minimum of three (3) current professional references to substantiate the vendor’s qualification are also required. The references must include the name, title, organization name, address, e-mail address and phone number. Do not use NYSED staff as references. Two points will be subtracted from the bidder’s score for incorrect or outdated reference information that prevents NYSED from contacting a reference.s

Financial Criteria (30 Points)

The RFP requires the submission of a cost proposal which must be submitted separately and labeled "Cost Proposal RFP #09-021 Do Not Open."

The bidder must submit a cost proposal reflecting the cost for the activities required for this RFP. The bidder must indicate the specific costs by completing the Bid Form Cost Proposal Schedule of Deliverables as well as the five year budget summary located in Submission Documents.

Each cost proposal submitted in response to this RFP must include (1) original plus (2) copies of all forms, certifications, and Bid Forms.

The Financial Criteria portion of this RFP will be scored based upon the grand total for the five year budget summary.

The financial portion of the proposal represents thirty (30) points of the overall score and will be awarded points pursuant to a formula which awards thirty (30) points to the proposal that reflects the lowest overall cost. The remaining budgets will be awarded points based on a calculation that computes the relative difference of each vendor’s cost proposal against the lowest budget. The resulting percentage is then applied to the maximum point value of thirty (30) points. This calculation will be computed by the Contract Administration Unit upon completion of the technical scoring by the technical review panel.

The agency reserves the right to request best and final offers.

Method of Award

The aggregate score of all the criteria listed will be calculated.

The contract issued pursuant to the proposal will be awarded to the vendor whose aggregate technical score and cost score is the highest among all the proposals rated.

In the event that more than one proposal obtains the highest aggregate score, the contract will be awarded to the vendor in that group of highest aggregate scores whose budget component reflects the lowest overall cost.

The NYS Education Department reserves the right to reject all proposals received or cancel this RFP if it is in the best interest of the Department.

Post Selection Procedures

Upon selection, the successful bidder will enter into negotiations for a contract with NYSED. The contents of this RFP, any subsequent correspondence related to final contract negotiations, and such other stipulations as agreed upon may be made a part of the final contract developed by NYSED. Successful bidders may be subject to audit and must ensure that adequate controls are in place to document the allowable activities and expenditure of State funds.

Appeals Process

If a bid proposal is not recommended for award, the bidder may file a formal written appeal of the award results within 10 business days of receipt of the notice of non-award.

The process is as follows:

The Contract Administration Unit (CAU) will convene a review team that will include at least one staff member from NYSED’s Office of Counsel, CAU, and the Program Office.  The appeal review team will review the written appeal, discuss the merits of the appeal and decide to approve or deny the appeal.  Counsel’s Office will provide the bidder with written notification of the review team’s decision within seven business days of receipt of the receipt of the appeal.  The original appeal and decision will be filed with OSC when the contract procurement record is submitted for approval and CAU will advise OSC that an appeal was filed.

The appeal and any documentation must be sent to:

Contract Administration
New York State Education Department
89 Washington Avenue, Room 505W - EB
Albany, New York 12234

Debriefing Procedures

All unsuccessful bidders may request a debriefing within 30 days of receiving notice from NYSED. Bidders may request a debriefing letter on the selection process regarding this RFP by submitting a written request to the Fiscal Contact person at:

NYS Education Department
Contract Administration Unit
89 Washington Avenue
Room 505W EB
Albany, NY 12234

The Fiscal Contact person will make arrangements with program staff to provide a written summary of the proposal’s strengths and weaknesses, as well as recommendations for improvement.

Vendor Responsibility

State law requires that the award of state contracts be made to responsible vendors. Before an award is made to a not-for-profit entity, a for-profit entity, a private college or university or a public entity not exempted by the Office of the State Comptroller, the Department must make an affirmative responsibility determination. The factors to be considered include: legal authority to do business in New York State; integrity; capacity—both organizational and financial; and previous performance. Before an award of $100,000 or greater can be made to a covered entity, the entity will be required to complete and submit a Vendor Responsibility Questionnaire.  School districts, Charter Schools, BOCES, public colleges and universities, public libraries, and the Research Foundation for SUNY and CUNY are some of the exempt entities.  For a complete list, see: http://www.osc.state.ny.us/vendrep/documents/vrdocrules.pdf external link icon.

Vendors are invited to file the required Vendor Responsibility Questionnaire online via the New York State VendRep System. To enroll in and use the New York State VendRep System, see the VendRep System instructions at: http://www.osc.state.ny.us/vendrep/systeminit.htm external link icon or go directly to the VendRep System online at https://portal.osc.state.ny.us/wps/portal external link icon.

For direct VendRep System user assistance, the OSC Help Desk may be reached at 866-370-4672 or 518-408-4672 or by email at helpdesk@osc.state.ny.us.

Vendors opting to file a paper questionnaire can obtain the appropriate questionnaire from the VendRep website: http://www.osc.state.ny.us/vendrep/templates.htm external link icon or will receive it with the award letter.

Note: Bidders must acknowledge whether they filed their questionnaire online by checking the appropriate box on the Response Sheet for Bids (Submission Documents).

Procurement Lobbying Law

Pursuant to State Finance Law §§139-j and 139-k, this solicitation includes and imposes certain restrictions on communications between the New York State Education Department ("NYSED") and an Offerer/bidder during the procurement process. An Offerer/bidder is restricted from making contacts from the earliest notice of the solicitation through final award and approval of the Procurement Contract by NYSED and, if applicable, Office of the State Comptroller ("restricted period") to other than designated staff unless it is a contact that is included among certain statutory exceptions set forth in State Finance Law §139-j(3)(a). Designated staff, as of the date hereof, is identified below. NYSED employees are also required to obtain certain information when contacted during the restricted period and make a determination of the responsibility of the Offerer/bidder pursuant to these two statutes. Certain findings of non-responsibility can result in rejection for contract award and in the event of two findings within a four year period, the Offerer/bidder is debarred from obtaining governmental Procurement Contracts. Further information about these requirements can be found at the following web link: http://www.oms.nysed.gov/fiscal/cau/PLL/procurementpolicy.htm.

Designated Contacts for NYSED
Program Office – (Alice D. King)
Contract Administration Unit – (Gary Yorkshire)

Consultant Disclosure Legislation

Effective June 19, 2006, new reporting requirements became effective for State contractors, as the result of an amendment to State Finance Law §§ 8 and 163.  As a result of these changes in law, State contractors will be required to disclose, by employment category, the number of persons employed to provide services under a contract for consulting services, the number of hours worked and the amount paid to the contractor by the State as compensation for work performed by these employees. This will include information on any persons working under any subcontracts with the State contractor.

Chapter 10 of the Laws of 2006 expands the definition of contracts for consulting services to include any contract entered into by a State agency for analysis, evaluation, research, training, data processing, computer programming, engineering, environmental, health, and mental health services, accounting, auditing, paralegal, legal, or similar services.

To enable compliance with the law, State agencies must include in the Procurement Record submitted to OSC for new consultant contracts, the State Consultant Services Contractor’s Planned Employment From Contract Start Date Through the End of the Contract Term (Form A external link icon - see link below). The completed form must include information for all employees providing service under the contract whether employed by the contractor or a subcontractor. Please note that the form captures the necessary planned employment information prospectively from the start date of the contract through the end of the contract term.

Form A: http://www.osc.state.ny.us/agencies/gbull/g226forma.doc external link icon

Chapter 10 of the Laws of 2006 mandates that State agencies must now require State contractors to report annually on the employment information described above, including work performed by subcontractors. The legislation mandates that the annual employment reports are to be submitted by the contractor to the contracting agency, to OSC and to the Department of Civil Service.  State Consultant Services Contractor’s Annual Employment Report (Form B external link icon - see link below) is to be used to report the information for all procurement contracts above $15,000.  Please note that, in contrast to the information to be included on Form A, which is a one-time report of planned employment data for the entire term of a consulting contract on a projected basis, Form B will be submitted each year the contract is in effect and will capture historical information, detailing actual employment data for the most recently concluded State fiscal year (April 1 – March 31).

Form B: http://www.osc.state.ny.us/agencies/gbull/g226formb.doc external link icon

For more information, please visit the OSC web site for G-Bulletin 226 at: http://www.osc.state.ny.us/agencies/gbull/g-226.htm external link icon.

Public Officer’s Law Section 73

All bidders must comply with Public Officer’s Law Section 73 (4)(a), as follows:

4. (a) No statewide elected official, state officer or employee, member of the legislature, legislative employee or political party chairman or firm or association of which such person is a member, or corporation, ten per centum or more of the stock of which is owned or controlled directly or indirectly by such person, shall (i) sell any goods or services having a value in excess of twenty-five dollars to any state agency, or (ii) contract for or provide such goods or services with or to any private entity where the power to contract, appoint or retain on behalf of such private entity is exercised, directly or indirectly, by a state agency or officer thereof, unless such goods or services are provided pursuant to an award or contract let after public notice and competitive bidding. This paragraph shall not apply to the publication of resolutions, advertisements or other legal propositions or notices in newspapers designated pursuant to law for such purpose and for which the rates are fixed pursuant to law.

(i) The term "state officer or employee" shall mean:
    (i) heads of state departments and their deputies and assistants other than members of         the board of regents of the university of the state of New York who receive no         compensation or are compensated on a per diem basis;
    (ii) officers and employees of statewide elected officials;
   (iii) officers and employees of state departments, boards, bureaus, divisions,         commissions, councils or other state agencies other than officers of such boards,         commissions or councils who receive no compensation or are compensated on a per         diem basis; and
   (iv) members or directors of public authorities, other than multistate authorities, public          benefit corporations and commissions at least one of whose members is appointed by         the governor, who receive compensation other than on a per diem basis, and         employees of such authorities, corporations and commissions.

Public Officer’s Law Section 73 can be found at http://www.nyintegrity.org/law/ethc/POL73.html external link icon.

Back to top

Assurances

Appendix A – Standard Clause for all New York State Contracts will be included in the contract that results from this RFP.  Vendors who are unable to complete or abide by these assurances should not respond to this request.

The documents listed below are included in Submission Documents, which must be signed by the Chief Administrative Officer.  Please review the terms and conditions.  Certain documents will become part of the resulting contract that will be executed between the successful bidder and the NYS Education Department.

Appendix A – Standard Clause for all New York State Contracts contains the following certifications:

  • Non-Collusion Certification
    (Signature Required - the form is included in Submission Documents)
  • MacBride Certification
    (Signature Required - the form is included in Submission Documents)
  • Certification-Omnibus Procurement Act of 1992
    (Signature Required - the form is included in Submission Documents)

Appendix A-1 – NYSED Contract Clauses

Certification Regarding Lobbying; Debarment and Suspension; and Drug-Free Workplace Requirements (Signature Required - the form is included in Submission Documents)

Offerer Disclosure of Prior Non-Responsibility Determinations (Signature Required - the form is included in Submission Documents)

Appendix A - Standard Clauses for NYS Contracts

Appendix A-1

Attachments

Back to top

Last Updated: November 2, 2009