Report Card Fixed Point Examination
Creating a Fixed Point
To Identify Agencies for Corrective Action
Issue – The New York State Report Card for Adult Education uses a quartile system to rank agency performance based on four variables: educational gain, the four follow-up measures, goal setting and survey response rate. For program year 2005-06 the lowest quartile of agencies are being placed under corrective action and must complete a corrective action plan. This involves approximately 50 agencies. It was recognized that, if the quartile system were to continue to be used, there would always be 25% of agencies under corrective action even if all or most agencies had exemplary performance. It was recommended that a fixed point of performance be set that, potentially, all agencies could meet. This point would be "proficiency" meaning that all agencies exceeding this point would be identified as "proficient" while those below would be identified as "not proficient".
What is proficiency? – A program performing below the State average would not be considered proficient by any definition. For this reason, it is proposed that the most recent State average in both educational gain and the aggregate of the 4 follow-up measures be used to set the benchmark for proficiency. In addition, since the USDOE established a minimum of a 50% survey response rate for those exiting the program with a follow-up goal, a proficient agency also must have an aggregate response rate of 50%. Since goal setting in employment and postsecondary education is a State priority, it is proposed that an agency can be proficient if the agency exceeds the benchmark in the previous three measures and meets one or both of the goal setting targets.
Identifying highly proficient, not proficient, and unacceptable agency performance – An agency that exceeds the proficiency benchmark and is in the top quartile would be rated exemplary. An agency that fails to meet the proficiency benchmark and is in the bottom quartile would be rated unacceptable. An agency that fails to meet the benchmark but is not in the bottom quartile would be rated not proficient.
Examples - In program year 2007-08, the most recent State average for educational gain is 36.4% and the State average for follow-up measures is 52.2% using data from for the 2005-06 program year. Agencies would be informed in July what the targets are to obtain a rating of proficient.
Example 1 - In the fall of 2008 it is determined that 85% of all agencies exceeded the proficiency benchmark. Only 15% of all agencies would be identified as unacceptable since 10% of agencies in the bottom quartile exceeded the proficiency benchmark, 60% of agencies would be identified as proficient, and 25% would be rated highly proficient. No agencies would be rated not proficient.
Example 2 – In the fall of 2008 it is determined that 65% of all agencies exceeded the proficiency benchmark. 25% of all agencies would be rated as unacceptable since they failed to meet the proficiency benchmark and were in the bottom quartile. 10% of all agencies would be rated not proficient because they failed to meet the proficiency benchmark but were not in the bottom quartile. 40% would be identified as proficient because they exceeded the proficiency benchmark but were not in the top quartile. 25% of all agencies would be rated highly proficient because they exceeded the proficiency benchmark and were in the top quartile.
Potentially, all agencies could be rated proficient or highly proficient in a given year.