On Jan. 16, a Joint Committee of the District 65 and 202 School Boards discussed the Districts’ Joint Literacy Goal which is to “ensure that all students are proficient readers and college and career ready by the time they reach 12th grade.”
Two days earlier, the District 202 School Board discussed a report that contained charts showing the percent of students who were proficient in reading when they entered Evanston Township High School using 1) the STAR test, which is given by ETHS; and 2) the MAP test, which is given by District 65.
The charts show the tests yield starkly different results.
For example, one chart shows that in the fall of 2018, 60% of all students who entered ETHS as freshmen were proficient in reading using the STAR test, while 84% were proficient in reading using MAP.
Another chart shows that in the fall of 2018, only 38% of Black students who entered ETHS as freshman were proficient in reading using the STAR test, while 66% were proficient in reading on the MAP test. See chart below illustrates the differences.
The report was prepared by Scott Bramley, Associate Principal for Instruction and Literacy, and Kiwana Brown, Reading Specialist, both at ETHS.
Members of the Joint Committee were frustrated that the results were so different, and that the Districts had not come up with a common assessment or a way to statistically align the STAR and the MAP tests. Several people questioned whether the tests and/or definitions of proficiency were aligned. See sidebar below.
Mark Metz, a member of the District 202 School Board, laid out his concerns. He said the School Boards adopted the Joint Literacy Goal in January 2014, and it was “frustrating” that the Districts have not yet developed metrics that were aligned to assess their joint progress toward meeting the goal.
He added that another concern was that the MAP test was not identifying many students who were not proficient in reading and who needed interventions. “We are under-identifying students who are not proficient in reading every year as they enter high school because all the data we [ETHS] have at that point is MAP. The fact of the matter is we’re getting MAP data only, so that’s how we’re placing them and then we’re discovering … we’re finding after the fact we’re under-identifying kids who need help – in big numbers. Then we’re having to go back and retrofit that and get them into special programs. We’re losing time.”
He said the MAP test is identifying about 120 students each year as lacking proficiency in reading, while STAR is identifying about 300 students who are not proficient in reading.
Mr. Metz said it is critical that ETHS be able to identify students who are not reading at grade level. “I’m looking for a sense of urgency to get this fixed … and resolve these data problems … to advance the goal of making sure each student by grade 12 is proficient.”
Dr. Bramley said, in his experience, the STAR results are more accurate than MAP in terms of making placement decisions.
Marcus Campbell, Principal of ETHS, said teachers “tell us consistently that STAR is far more reliable because it’s what they see on the ground as they teach every day.”
Gretchen Livingston, a member of the District 202 Board, said, “There needs to be a sense of urgency around this. … We need to have a better plan and it needs to happen now.” She suggested the Districts could have an interim plan and then develop a long-term plan later.
Pete Bavis, Assistant Superintendent of District 202, said there are three ways to address this:
1. Districts 65 and 202 could adopt a similar
2. ETHS is using the SAT suite of assessments for grades 9-12, and District 65 could use the
SAT suite of assessments in grades 7 or 8 and
then backmap to earlier grades, and
3. The Districts could do a statistical alignment between STAR and MAP, which he said may or
may not work.
Dr. Bavis said the first two options are pretty simple solutions. The third requires a lot of time.
District 65 Superintendent Paul Goren said that District 65 is currently giving many tests, and if a new assessment was added, the District would need to eliminate one that is currently being used. He said, “Part of our work over the next year plus is to do a deeper dive into what our assessment system looks like and how we might change it.”
He said based on his work at the Chicago Public Schools, he thought MAP seemed to be a better assessment to help teachers frame instruction, and added that it would be a “medium to a major lift” to add STAR because it was a computer adaptive test and “all of our kids don’t have immediate access to computers.”
Stacy Beardsley, Assistant Superintendent of Curriculum and Instruction for District 65, said a study done by Renaissance, the owner of the STAR test, concluded there was a high correlation between the STAR test and MAP in both reading and math. Assuming the tests are correlated, the two research departments could discuss if there are cut scores on the tests that would provide a consistent benchmark for measuring proficiency. She said, “This is probably the direction that I would want to step first before trying to decide if we do more assessment.”
Dr. Beardsley also said that the District has 15 years of MAP data, and cautioned that when assessment systems are considered, the assessment system needs to provide tools that help teachers make really strong instructional decisions that will benefit students.
“If it doesn’t inform teaching and learning well, then we will have got something that the Board can use, but not our teachers and students,” said Dr. Beardsley.
Jonathan Baum, a member of the District 202 School Board said, “The vast majority of school districts in the nation are K-12. How do they do this?” He asked administrators to research what K-12 districts are doing.
District 202 Board President Pat Savage-Williams said, “We need to move as quickly as possible,” adding that coming up with Option 3 may take another year.’
District 65 Board Vice President Anya Tanyavutti said in addition to discussing possible ways to align proficiency levels, the administrators should determine “what are the implications of adopting another assessment tool, and not just what they perceive, but run that by stakeholders, including students.” She said the Districts should move with urgency, but thoughtfully.
Dr. Bramley said, “What we really need to consider is do we want to have a measure of reading proficiency and growth for grades 3-12. And if we do, we need to adopt a system that will allow us to do that. If we’re going to use different measures, I think we’re going to continue having these frustrations.”
Dr. Goren said before the Joint Board meeting on Feb. 25, the Districts will get their research teams together and continue the conversation, and attempt to come up with a short-term solution and a long-term solution.
At the Joint Committee meeting on Jan. 16, Stacy Beardsley, Assistant Superintendent of District 65, said she was not in a position to comment on the different results shown for STAR and the Measures of Academic Progress (MAP) test because she lacked information concerning the definitions of proficiency used for each test and also did not know what MAP test data was used. She said ETHS uses the highest score an eighth-grader earns on the Fall, Winter or Spring MAP tests in making a placement decision. She said she was not sure if the data presented in the ETHS report used the highest score of the three MAP tests or just the score on the Spring MAP test.
She added there was a significant difference in results between the Fall and Spring MAP tests, and “We see a drop anywhere from 15% to 30% from the Fall scores over meeting college readiness benchmarks.” She said this could be due a lack of motivation of eighth-graders.
But the drop could also be due to a drop in academic achievement.
Scott Bramley, Associate Principal for Instruction an Literacy at ETHS, told the RoundTable:
• In reporting the percentage of ETHS freshmen who are proficient on the STAR test, ETHS used a grade equivalent score of 8.3 as the benchmark to measure proficiency. If a ninth-grader received a grade equivalent score of 8.3, that means that the student scored as well on the STAR Reading test as a “typical” student in the third month of eighth grade.
• In reporting the percentage of students who were proficient on the MAP test, ETHS used the highest score a student had received as an eighth-grader on the Fall, Winter or Spring MAP test. If that score was above the 50th percentile for the years 2014, 2015 and 2016, the student was regarded as proficient. If that score was 220 or above in the years 2017 and 2018, the student was regarded as proficient. An eighth-grade score of 220 in reading on MAP equates to the 50th percentile on the Spring MAP test.
A score at the 50th percentile – which is the median – could be considered a typical score.
While the MAP test is generating significantly higher results than the STAR test, the RoundTable recently reported that students in the Chicago Public Schools were showing significantly higher norm-based results on the MAP test than on several other tests.
For example, 73.5% of CPS eighth-graders ostensibly scored at or above the national average score in reading on the Spring 2017 MAP test. In contrast, only 42% of CPS eighth-graders scored at or above the national average score in reading on the 2017 National Assessment of Education Progress test. The analysis was conducted with Paul Zavitkovsky, a researcher and leadership coach at the Center for Urban Education Leadership Program at the University of Illinois at Chicago.