At their April 12 meeting, District 65 School Board members discussed the District’s 2020 Accountability and Achievement Report. Due to limitations resulting from the COVID-19 pandemic, the report provides more limited data: the percentage of students meeting the 50th percentile on the Measures of Academic Progress (MAP) test given in the winter of the 2019-20 school year and two prior years, and the percentage of students meeting “expected gains” on the same test. [1]
“We recognize that this does not display a full picture of the 2019-20 school year,” Kylie Klein, Director of Research, Accountability, and Data, told the RoundTable. “The disruption of the pandemic really limited our ability to show the full range of student learning through the spring. We know that our teachers and our schools continued to provide learning opportunities to our students that are not captured by this data.”
All of the data reported is pre-pandemic. The data does not show the impact of the pandemic on student learning.
One section of the report – a qualitative section – celebrated accomplishments of students, educators, and principals and contributions by the community. The RoundTable reported on that section of the report in a prior article, available here.
Some Caveats re: Standardized Test Data
In a study published in September 2019, Sean Reardon, Professor of Poverty and Inequality in Education at Stanford University, and his co-authors say, “We examine racial test score gaps because they reflect racial differences in access to educational opportunities. By ‘educational opportunities,’ we mean all experiences in a child’s life, from birth onward, that provide opportunities for her to learn, including experiences in children’s homes, childcare settings, neighborhoods, peer groups, and their schools.
“This implies that test score gaps may result from unequal opportunities either in or out of school …”
District 65’s report says, “When aggregate data show that members of a particular student demographic group score below benchmarks, these outcomes reflect opportunity gaps faced by marginalized groups due to institutional racism in the education system and for many families insufficient social and economic supports. These results should not be used to draw conclusions about the efforts, abilities, or strengths of these students or their families.”
In that spirit, the scores reported in this article measure how well School District 65 and the Evanston community, as a whole, are providing equitable opportunities to the children in Evanston.
Percentage of D65 Students At/Above the 50th percentile
The charts below show the percentage of District 65 students who scored at or above the 50th percentile on the MAP tests in English Language Arts (ELA) and Math given in the Winter of the school years 2017-18, 2018-19, and 2019-20. [2]
“Students at the 50th percentile are performing at the typical, or average, level for students in their grade,” says the report.
Performance at the 50th percentile is often equated to performing at grade level.
Ms. Klein said in a memo to the School Board, “The three-year trend has been generally stable for both ELA and Math. The data shows significant gaps by race and demographics.”
The data shows for the MAP tests given in the 2019-20 school year:
- In ELA, 43% of Black students and 48% of Hispanic students scored at/above the 50th percentile, compared to 90% of White students, 79% of Asian students, and 83% of multi-racial students;
- In math, 32% of Black students and 46% of Hispanic students scored at/above the 50th percentile, compared to 86% of White students, 77% of Asian students, and 76% of multi-racial students.
The proficiency level needed to meet the 50th percentile is lower than the proficiency level needed to meet the college readiness benchmark scores identified for the MAP test by the Northwest Evaluation Association (NWEA), the owner of the test. Those benchmark scores correspond, on average, to the 63rd percentile in reading, and to the 68th percentile in math.
A joint study recently conducted by School Districts 65 and 202 concluded that in order for a student to be viewed as “proficient” in reading when entering freshman year at Evanston Township High School, the student must score at or above the college readiness benchmark scores identified for the MAP test.
Making Expected Gains
The report also provides the percentage of District 65 students who made “expected gains” in ELA and math on the MAP test given in the winter of the 2019-20 school year. [3]
Conceptually, to make expected gains, a student must grow academically during a school year as much as or more than the average student in their grade who started out at the same achievement level. Statistically, on a nationwide basis, one would expect that 50% of the students will meet expected gains using this approach.
Overall, 62% of District 65 students met expected gains in ELA and 52% met expected gains in math.
The chart below shows the percentage of students, by subgroup, who made expected gains on the Winter MAP test in 2019-20.
The differences between subgroups are not as pronounced as in the charts showing the percentages of students meeting the 50th percentile. For example:
- In ELA, 57% of Black and Hispanic students made expected gains, compared to 66% of White students, 72% of Asian students, and 64% of multi-racial students.
- In math, 47% of Black students and 48% of Hispanic students made expected gains, compared to 55% of White students, 61% of Asian students, and 59% of multi-racial students.
The data shows that each subgroup is growing academically. But Black and Hispanic students are not showing the accelerated growth needed to reduce the wider gaps in the percentages of students meeting the 50th percentile (or meeting college readiness benchmarks).
A recent analysis conducted by the RoundTable and Paul Zavitkovsky, Assessment Specialist at the University of Illinois at Chicago’s Center for Urban Education Leadership, shows that many fifth graders will not meet NWEA’s college readiness benchmarks by the end of eighth grade if they just meet NWEA’s expected gains in sixth, seventh, and eighth grades.
Mr. Zavitkovsky concluded, “For any student achieving at less than 1.0 to 1.5 years above grade level in the spring of fifth grade, limiting expectations to NWEA growth targets reinforces the status quo and perpetuates opportunity gaps that currently exist at District 65.” The analysis is available here.
NWEA acknowledges this point in NWEA’s 2020 Growth Study (July 2020). It says, on pages 84-85, “The individual student growth projections provided in NWEA reports represent average growth (i.e., 50th percentile conditional), relative to the norms, but provide no assurance that students will meet state-defined proficiency or other standards by demonstrating that level of growth. … The norms provide information about average achievement and growth for U.S. students but provide no information about necessary or sufficient achievement or growth in any U.S. state relative to grade-level proficiency or college readiness standards or aspirational growth goals.”
The RoundTable asked Ms. Klein why the District continued to use NWEA’s expected gain targets when meeting those expected gains, in essence, maintains the status quo and perpetuates opportunity gaps.
Ms. Klein told the RoundTable, “There’s a lot of different metrics that we use. That’s not the goal. We want to go, of course, further than that. This is one metric we are using to help inform the community. It’s not the only metric. We’re always looking for other opportunities to improve student outcomes.”
NWEA says, on page 84, that school districts may set “customized growth goals aligned to state proficiency and college readiness benchmarks.”
Customized goals could set accelerated growth targets for many students. School District 65 has not done so.
When Will the District Present Data Showing the Impact of the Pandemic?
Board member Soo La Kim asked if the District would have enough data at the end of the spring to assess if there is any significant difference between the online and in-person learning.
Ms. Klein said, “I think that’s a really important question.” She said the District did multiple different analyses to make sure that the testing results of the assessments that were done online were meaningful and reliable, and added, “I think that we are going to be doing a lot of applied internal research and analysis of these different data that we’ve collected this year to really better understand student outcomes.
“A full picture of how this pandemic has affected our outcomes may take several years to fully understand, both in our District and nationally,” said Ms. Klein.
She did not give a time when the District would provide data showing the impact of the pandemic on student learning, or which would assess which students or subgroups of students needed interventions.
Using Data to Inform Instruction
Stacy Beardsley, Assistant Superintendent of Curriculum and Instruction, said, “we made a very strong commitment” to administer the MAP test three times this year. Teachers were expected to use the assessments to plan their instruction to meet the learning needs of students, and also to assess whether students learned what they were expected to learn.
“So, the pre and the post gives educators some information about where kids are in those skills, and helps to inform the planning,” said Dr. Beardsley. “And then the post gives us information about how successfully our instruction was. We can continue to use that information to inform interventions going forward, and also professional learning and improvement,” she said.
Andalib Khelghati, Assistant Superintendent of Schools, said this year the District has been building professional learning communities (PLCs) in the schools. The PLCs include teachers and administrators who meet regularly to review and improve practices and procedures.
“The PLC work is really thinking about what the curriculum is asking for, and then going a step further, which is, what do we want our students to understand? What are we looking for them to achieve in that period of time? How are we going to measure it? And then what happens when students are not achieving it?”
Dr. Khelghati said that in some of the informal initial feedback from principals, “we can see that there is a high level of attention to, let’s say, the social-emotional learning environment. A lot of the principals have talked about the way in which they see relationship-building happening in the classroom space.
“On the flip side, there are areas for opportunity, the focus on standards and the objectives of those lessons. How do we really accelerate the learning that’s happening in the classroom?
“These are just a few of the trends that are emerging.”
Dr. Khelghati said the PLCs have enabled principals to start to see the instructional needs in the schools, which in turn can be used to determine what professional development should be provided.
Terrance Little, Assistant Superintendent of Secondary Schools, said another advantage of the PLCs is that principals can see if instruction is keeping up with the pacing needed to complete a full year’s worth of instruction within the school year. “We’re seeing a good pacing, and everybody’s kind of keeping up with the pacing guides, and it’s coming from working together and being coached by our presence.”
When asked about the interventions taking place, Dr. Khelghati said, “I don’t have a comprehensive analysis. I do know that many principals are focused on small group instruction as part of their intervention strategy. I do know that small group instruction has been a part of what the building coaches have been focusing on.”
Administrators and teachers are also collaborating in school “Reality Check” meetings. These meetings, held three times during the school year, provide school administrators and teachers with opportunities to discuss a wide range of data and topics with various administrators. The goal is to generate ideas on how to build on student’s strengths and provide supports.
The Theory of Change
Board President Anya Tanyavutti asked what “our theory of change is? We’ve certainly seen a lot of shifts in practices,” she said, “But I would like to see the shifts in outcomes too, as I’m sure we all would.”
Latarsha Green, Deputy Superintendent, said, “I think what we’ve been looking to achieve is some type of a comprehensive movement in a direction towards transformative change. …. I think what we are engaging in now is somewhat of a doubling down in some practices that are not new, not within themselves capable of helping us to transform this disproportionality that we’ve seen. But I do think it speaks to a comprehensive plan within our miracle system, and speaking specifically to a system to support the different mechanisms needed in change.”
“I would say more so than anything, is trying to really just coordinate those resources into a forthcoming strategic plan,” said Dr. Green.
Board member Biz Lindsay-Ryan focused on “elevating and escalating the skills of the teachers who are doing it right and using them as coaches and mentors, because we do have some people who are having phenomenal progress. … We need to get as many people connected with this excellent educator as possible and get the mentoring and coaching going. “
Dr. Khelghati said the District has been attempting to support school leaders to improve instruction in the classroom through coaching and consistent daily and weekly feedback to teachers, which, he said, teachers have been asking for. “Instruction is really argued to be a leading indicator for outcomes and achievement,” he said.
Dr. Khelghati said the change is going to come where principals “really develop their coaching skills,” and support teachers, and then “connecting that to a school improvement planning process and the professional learning that has to happen to accelerate that work.”
Ms. Lindsay-Ryan said, “It may not always be that you have the right coach and mentor support in your building.” She suggested the District take advantage of teachers in other buildings who are getting good results. She said she wanted to make sure “that’s part of our larger District vision.”
Dr. Green said, “I’m excited to see us move forward next school year with instructional rounds … led by teachers to peer observe and see what each other are doing. … We might as well look and learn from each other.
“I hope we capitalize on the technology that we’ve had to use this year, to figure out how we could have stronger collaboration across buildings, even if it’s educators recording their lesson, and somebody’s watching it, not live, and giving feedback, or being able to see what you know, particularly as we talk about some of these curricular shifts and equity. Let’s see how the teacher at Kingsley in third grade is addressing the Black Lives Matter curriculum and use that as a tool and a template for other third grade teachers, if we have somebody knocking it out of the park.”
Board member Joey Hailpern said, “I would still love to see in the strategic plan world, Dr. Green, ‘here’s our plan for the next five years as we try to tackle these practices institutionally with expected data points along the way that we can then in this report, report on.’ Are we with that target or not? Are we ahead of it? And if we are great. If we’re not, here’s why, knowing that those goals might be beyond the scope of reasonable prediction.
Footnotes:
[1] The report says the pandemic has limited the District’s ability to present data. Schools were closed for in-person learning in March 2020. One significant limitation was that the Illinois State Board of Education did not administer the Illinois Assessment of Readiness in the Spring of 2020. In addition, due to the pandemic the District did not administer the Measures of Academic Achievement (MAP) test in the Spring of 2020.
As a result, the District decided to report student test results from the MAP test given in the Winter of the 2019-20 school year, and in two prior years.
The District also decided to report the percentage of students who met the 50th percentile on the MAP tests given in the Winter because the college readiness benchmark scores for the MAP test are applicable for the MAP test given in the Spring.
[2] In the 2019-20 school year, the Winter MAP test was given to fourth through eighth graders in ELA and to second through eighth graders in math. Ms. Klein cautioned that the data for the 2017-2018 school year is incomplete because not all District 65 schools administered the Winter MAP test in that year.
[3] Student growth data is shown for students who have MAP scores from both school year 2018-19 and school year 2019-20, therefore, for students in grades 4-8 in ELA and grades 3-8 in math.