At the District 65 School Board’s Aug. 19 meeting, John Gatta, a principal of the ECRA Group, presented a revamped teacher appraisal system worked out by the District 65 Teacher Evaluation Committee (TEC). The new system lays out how student growth will be taken into account in evaluating teachers. TEC, a committee composed of 10 administrators and 10 teachers, began meeting in January 2013 to discuss this contentious issue, and it reached a consensus in June. ECRA, an educational research firm, provided technical support.
Former Superintendent Hardy Murphy signed off on the new teacher appraisal system in early June.
The teacher evaluation system contains a student growth component and a professional practices component. The new document focuses on the growth component, which it says “will make up 50% of a teacher’s summative rating.” In practice, though, the higher of the growth or professional practice rating will often control.
“This is a far more sophisticated system and is a far more equitable system” than the current one, said Dr. Gatta. “It’s designed with all the current thinking on how to do this.” ECRA will be retained to do all the calculations required under the new model.
Growth of Three Student Groups
Under the new model, each teacher will be evaluated based on the growth of three different groups of students:
• All Students in a Teacher’s School: 10% of a teacher’s evaluation will be based on the growth of all students in a teacher’s school in reading and math. This component will look at growth in the prior school year, with variations for new teachers.
• A Teacher’s Roster of Students: 10% of a teacher’s evaluation will be based on growth of all students in a teacher’s roster in reading and math. This applies regardless of the subject taught by the teacher. For example, 10% of an art teacher’s evaluation will be based on how his or her students improve in reading and math.
• The Teacher’s Class: 30% of the teacher’s evaluation will be based on the growth of students taught by the teacher in the subject(s) he or she teaches. Growth will be measured using MAP, PALS or internal assessments, CBAs.
Measuring Value Added Growth
On a big picture basis the new model will measure “value added growth” by comparing each student’s projected growth with his or her actual growth. The difference is either value-added by a teacher, or value-lost, which is the basis for assessing teachers.
Projecting Each Student’s Growth. To measure student growth, ECRA will first project what each student’s growth will likely be in the coming school year “based on student propensities derived from individual student performance on multiple past assessments.”
In other words, Dr. Gatta said, ECRA will look at how a student has done in the past and project how he or she will do in the coming year. The past assessments may include a student’s achievement on the Illinois Standard Achievement Test (ISAT), the Measures of Academic Progress (MAP) test, or a CBA (community based assessment).
Dr. Gatta said to the extent a student’s demographics (such as income status or disability) impact achievement, it has impacted the student in the past and is thus automatically factored into the projections of individual student growth.
The model does not identify precisely the past assessments that will be used in projecting each student’s growth or the formula that will be used in making the projections.
Measuring Value Added Growth. At the end of the school year, ECRA will calculate the difference between each student’s projected growth and his or her actual growth. The difference will be converted to a standardized, common scale using “growth percentiles” to arrive at “Value Added Growth (VAG),” Dr. Gatta told the RoundTable.
If the actual growth of a student is equal to his or her projected growth, then the VAG for the student will be zero. This is because the student would have grown exactly as expected, and there would be no value added growth.
Under the model, a teacher whose groups of students have a VAG of zero will be rated “proficient.” Proficiency is thus defined in terms of achieving projected growth (or maintaining a student’s historical trend), rather than accelerating a student’s growth.
Dr. Gatta said while this is a fair way to evaluate teachers, “it may not be what we ultimately desire from the system or what we desire out of individual students. If all kids continue grow at rates that are typical or as we would have expected, then, guess what, we’re not going to close the achievement gap.”
Mr. Gatta said the District could focus on accelerating growth through other policies.
Significantly, the model also provides a “range” of VAG scores for assessing whether a teacher is “proficient.” Dr. Gatta said the “range” is plus or minus 0.3 (30%) of the standard deviation. He said this formula is “based on research findings that this value indicates educational relevance.”
In response to RoundTable questions, Dr. Gatta did not define how large plus or minus 0.3 of the standard deviation was in terms of a student’s projected growth or in terms of ISAT scale scores or MAP Rit scores. He told Board members, though, “The system is really designed to just capture the really, really good ones [teachers]and the really, really, really poor ones. The vast majority of teachers on the growth performance are going to land in the green room [“proficient”] – and that was by design. And that’s appropriate in a District like Evanston.
If the average VAG of a teacher’s students is within the “range” (between +.30 and -.30 of the standard deviation), then the teacher is deemed to be “proficient” for purposes of the student growth rating. If the average VAG is above that level, the teacher is deemed “excellent.” If the average VAG is below the “proficient” level, then the teacher is deemed either as “needs improvement” if the average VAG is between -.30 and -.60, or “unsatisfactory” if the average VAG is less than -.60.
A teacher’s summative rating is based on a combination of the overall student growth rating and the overall teacher practice rating. The new model contains a matrix that determines how the two ratings will be combined. If a teacher receives the same rating under both the growth and the practice models, the teacher’s summative rating is that rating. For example if a teacher’s rating is “proficient” using the growth model and “proficient” using the practice model, the teacher’s summative rating is “proficient.”
If, however, the teacher’s rating is different using the growth and practice models, the teacher gets the benefit of the doubt, and generally gets the higher rating, or a rating between the two. For example, if a teacher is given a rating of “needs improvement” in the growth model and a “proficient” rating in the practice model, the teacher’s summative rating is “proficient.” If a teacher’s rating is “needs improvement” in the growth model and “excellent” in the practice model, the teacher’s summative rating will be “proficient.”
Paula Zelinski, president of the District Educators Council (DEC, the teachers union), said teachers on TEC came to consensus on the model. She said teachers were informed about the new model at the end of the last school year. “I’m not sure they are all aware. … We’re interested in seeing how this rolls out in the buildings – how principals present this.” She added, “We see this as far more valid than the prior system.”
After hearing Ms. Zelinski’s comments, School Board member Richard Rykhus said, “I’m really happy to hear that.” He added when teachers opposed a prior appraisal system last November, “we had a roomful of teachers telling us there wasn’t support.”
Claudia Garrison explored the avenues a teacher would have to appeal a rating under the system. Tracy Quattrocki asked if it made adjustments for the non-random assignment of students, which had been raised as a concern last fall.
Board member Candance Chow and Mr. Rykhus said they were glad to see that teachers would be rated in part based on the growth of all students in the school. They said this would foster collaboration.
The District has had a teacher appraisal plan that takes student growth into account since 2010. That model essentially evaluated teachers based on whether more students scored above the 50th percentile at the end of a school year than at the start.
During April and May 2012, the administration proposed several new models that DEC did not agree to. Nonetheless, in late August 2012, then Superintendent Hardy Murphy unilaterally imposed a new appraisal system on teachers for the 2012-13 school year.
Administrators presented the new appraisal system to the School Board in an open meeting for the first time on Nov. 6, 2013. More than 200 teachers packed the Board’s meeting room at that time to demonstrate their opposition. DEC’s leadership said the methodology used to measure student growth under the that system was unreliable, statistically flawed and inequitable for a host of reasons. Dr. Murphy said there was no reason to delay implementing the new appraisal system and defended it, saying safeguards were built into the system.
On Dec. 3, the School Board decided it would vote on whether to defer implementation of the appraisal system until the beginning of the 2013-14 school year. Four Board members said they were concerned that system was statistically unreliable because it was based on extremely small groupings of students and did not take into account the non-random assignment of students, as well as other reasons.
On Dec. 17, Dr. Murphy announced that he had reached an agreement with DEC to treat the appraisal system as a shadow system for the year. He said this would allow the District to review and statistically verify the model. Dr. Murphy and DEC subsequently agreed to use ECRA to provide technical assistance in designing the new model.
Teacher summative ratings are important for track movements, which are one component in determining a teacher’s compensation. A teacher is eligible to move up one track every three of four years. In order to move up one track, teachers must receive at least two “”excellent”” ratings in the three or four-year period, and no rating below “”proficient.”” There are other requirements, as well, relating to continuing education and leadership participation.
If a teacher moves up a track, he or she receives an increase in compensation.