By Mary and Larry Gavin
At a Joint Committee meeting on Jan. 16, representatives from School Districts 65 and 202 appeared to be more at ease than they were nearly a year ago in discussing how they would measure and present to the public progress on their Joint Literacy Goal.
The meeting was a planning meeting for the Feb. 24 Joint Board meeting.
The joint literacy goal is to “ensure that all students are proficient readers and college and career ready by the time they reach 12th grade.”
Representatives from both School Districts said at the planning meeting they felt they had ironed out the problems and had responded to the concerns expressed by the members of both School Boards last year.
At a District 202 School Board meeting on Jan. 14, 2019, District 202 administrators presented a report to the District showing that 16% of the students entering ninth grade at Evanston Township High School in 2018 were not reading “proficiently,” according to the Measures of Academic Progress (MAP) test, but that 40% were not reading “proficiently,” according to the STAR test.
District 65 uses the MAP test, and ETHS, the STAR test.
An important issue is whether the number District 65 eighth-graders not reading proficiently when they entered ETHS was 16% or 40%.
The report, prepared by Scott Bramley, ETHS Associate Principal for Instruction and Literacy, and Kiwana Brown, Reading Specialist, provided five years of data, disaggregated by race and defined “proficiency” in terms of grade-level performance or typical performance at a specific grade level.
For the MAP test, the report defined proficiency as scoring at or above the 50th national percentile (or a MAP RIT score of 270) on the spring MAP test in eighth grade, which is often used as an indicator of grade-level performance. For the STAR assessments, proficiency was defined as scoring above a grade-equivalent score of 8.3 shortly after students enter ninth grade. STAR says an 8.3 is a typical score for a student at the third month of eighth grade.
At a Joint Committee meeting of District 65 and 202 School Boards exactly one year before – Jan. 16, 2019 – representatives of both districts discussed the issue and asked administrators to consider adopting the same test or to statistically align the proficiency levels of MAP and STAR so the Boards could monitor progress in meeting the Joint Literacy Goal.
On Feb. 25, 2019, at a Joint School Board meeting, Stacy Beardsley, Assistant Superintendent of Curriculum and Instruction at District 65, and Pete Bavis, Superintendent of Curriculum and Instruction at District 202, said they reached an agreement on how to track progress toward meeting the Joint Reading Goal. While the school districts agreed on how to measure progress toward the Joint Literacy Goal, they each decided to retain their current assessments – MAP in District 65 and STAR at ETHS. Neither district analyzed which test more reliably measures whether students are reading at grade-level.
The Jan. 16, 2020, Planning Meeting
Refinements of how the Joint Literacy Goal would be measured, given the two methods of assessment, was the centerpiece of discussion at the Jan. 16, 2020, meeting, held at 8 a.m. at Evanston Township High School.
At the Jan. 16 planning meeting, Dr. Beardsley said, “Since last February, the Districts 65 and 202 curriculum and instruction and research departments have been working together to follow up on the discussion where we agreed upon a set of benchmarks across essentially third grade through 12th grade, essentially across MAP and STAR.
“So we have been working on agreeing upon generating a set of tables that will constitute the report and the progress toward the goal.
“The intention is so share the data from the last school year as a model, so that we can 1) look at the data and 2) look at and understand how we are saying [how] we are seeing and reporting the data. At that point, we will agree on a format that will work, as we go into next year, in providing historical data so we can look at progress toward the goal.”
District 202 Board member Gretchen Livingston asked Dr. Beardsley to elaborate on the difference between the MAP test, given by District 65, and the STAR assessments.
“Can you elaborate on how that’s been handled?” she asked.
Dr. Beardsley responded, “Essentially, we took a look at what would be the appropriate cut points, and for District 65, we agreed it is the college-and-career readiness benchmark. For District 202, it is the grade equivalent for STAR. We did pretty much work and took a look at two cohorts, using those cut points, at what students would achieve in both.”
“We took a sample cut from District 65 and District 202 – took a simple cohort of students, mapped it out and arrived at what we agreed upon as being a fair and appropriate set of cuts,” Dr. Beardsley continued.
“We know that there’s some error there … but what we could see is there’s a pretty high degree of overlap when students achieve in both, and then we could learn also from the students who achieved with STAR or MAP.
“We came to an agreement as what these cut points would be last year, and we generated reports,” she said.
Mr. Bramley said, “We’re going to give a snapshot at [the February Joint Board meeting] with that structure. … The report will be a model for moving forward.”
“And you’re happy with it?” asked Ms. Livingston. “So the idea is that this is responsive to the concerns of both Boards at the last joint meeting?”
“It seems to be the best way that we can help the community understand where we are in terms of the goal, given where we are with the achievement methods,” Mr. Bramley responded.
Ms. Livingston continued, “I would be interested in hearing a little bit about the methodology and the basis for the methodology.” She said she would like to know whether the methods chosen for the two Districts had “a basis in research – not just your own research but research more broadly.”
District 202 Superintendent Eric Witherspoon said, “This meeting is not about measuring the goal; it’s about how we’re going to measure the goal.”
“We want to make sure we’re doing what’s expected, because in years past there’s been some level of concern that we haven’t been meeting that agenda, and so we went back to work. And so we want to make sure that this met the goal for both Districts and that it makes sense for the community, and the community accepts it, and we can move forward and use that as a model,” Mr. Bramley said.
“We have data for transition from eighth grade to ninth grade for the class of 2022,” he added.
Ms. Livingston asked if there is a way “to go back in time – take this model and go back?”
“I’m guessing we can go back, like even a year or two before [we adopted] the goal, to get a baseline and bring that data forward” said Dr. Witherspoon.
“I think that would be even better,” said Ms. Livingston.
Pete Bavis, Assistant Superintendent for Curriculum and Instruction at ETHS, said, “This is not a perfect world. In a perfect world, we’d have unified data. This is Perfect World 2. We have to be mindful of that. We have two separate measures that measure separate things.
District 65 Board President Pat Savage-Williams asked, “Is it too soon for us, regarding the incoming freshmen, to start looking at that?”
“Not too early,” said Dr. Beardsley.
“For how many years do we have STAR data?” Dr. Witherspoon asked Mr. Bramley.
“As soon as we implemented the goal, we started doing STAR,” Mr. Bramley responded.
“So we may not have data prior to STAR,” said Dr. Witherspoon, but where STAR begins, we’ll bring it forward.”
Ms. Livingston said, “My expectations are high. I don’t want to understate this. We’ve spent almost a year on his. It might knock our socks off. I anticipate that we’ll all be excited.”
“No pressure or anything,” she added.
Dr. Witherspoon said, “I will acknowledge we haven’t had the smoothest discussions on this.”
“Moving forward,” said Ms. Savage-Williams, “how can we use this to inform our work for the next group of students? ‘This is what we have. This is how we’re measuring it.’ How can we use this?”
Ms. Livingston said, “We need to be upfront and clear in this very involved Evanston community, which needs to understand what we’re doing.
District 65 Board member Joey Hailpern asked about the early levels, pre-K through third grade. “This is our one chance to look at it communitywide, and get that the 8th-9th transition is the focus, but I would like another supplement.”
Dr. Beardsley responded that there are “a couple of pieces to that.” The first, she said was that the current test is six years old and did not have, for them, the proper cut points. The second piece is that the District is implementing a new one-to-one literacy assessment and they “would like to work through that process.”
She added, “We will still absolutely record the one-to-one literacy outcomes in grades K-3. We will also have the MAP outcomes. We see some positive indicators of good work and growth, but, because of where we are right now, I want to wait.”
Mr. Hailpern said he appreciated the explanation and added he would like a placeholder for that work. He also asked for an update at the Joint Board meeting “and a placeholder [in the report] that it’s not being reported on because of the [focus on] transition [from eighth grade to high school].
District 65 Board member Sergio Hernandez said he wanted to make sure that there is continuity in the transition for black and brown families.
Dr. Beardsley said, “I hear the interest in continuity. I hear it and we’re going to do the work on it and be sure in the transition we have the type of rigorous assessment in place.”
Discrepancies Between MAP and Other Tests
While the MAP test is generating significantly higher results than the STAR test, the RoundTable reported in an article in August 2018 that students in the Chicago Public Schools were showing significantly higher norm-based results on the MAP test than on several other tests. The study is available here:
For example, 73.5% of CPS eighth-graders ostensibly scored at or above the national average score in reading on the Spring 2017 MAP test. In contrast, only 42% of CPS eighth-graders scored at or above the national average score in reading on the 2017 National Assessment of Education Progress test. The analysis was conducted with Paul Zavitkovsky, a researcher and leadership coach at the Center for Urban Education Leadership Program at the University of Illinois at Chicago.