At the joint meeting of the District 65 and 202 School Boards on Feb. 25, top administrators of each School District, presented their plan on how to consistently measure progress in meeting the Joint Literacy Goal adopted by the School Boards in January 2014.
The Joint Literacy goal is: “District 65 and District 202 will ensure that all students are proficient readers and college and career ready by the time they reach 12th grade.”
How to measure progress in meeting the goal consistently across the K-12 span has been a challenge, because the Districts use different tests. ETHS uses the STAR assessment, and District 65 uses the Measures of Academic Progress (MAP) test.
A year ago, in February 2019, a Joint Committee of the School Boards discussed a report showing the percent of students who were proficient in reading when they entered Evanston Township High School using: 1) the STAR test, which is given by ETHS; and 2) the MAP test, which is given by District 65.
In addition to the tests being different, different definitions of proficiency were used for each test.
The data presented at that 2019 meeting showed the tests and the different definitions of proficiency yielded starkly different results. For example, one chart showed that in the fall of 2018, only 38% of black students who entered ETHS as freshman were proficient in reading using a STAR test given in September, while 66% were deemed proficient in reading a few months before using the Spring MAP test.
Members of the Joint Committee were frustrated that the results were so different, and asked that administrators come up with a common assessment or a way to statistically align the STAR and the MAP tests.
On Feb. 25, 2020, administrators advised the Boards that they had come up with a way to measure proficiency in reading consistently across the two tests.
A written report summarizing the study was prepared by Carrie Levy, Director of Research, Evaluation and Assessment at Evanston Township High School; Pete Bavis, Assistant Superintendent of Curriculum and Instruction at ETHS; Kylie Klein, Director of Research, Accountability, and Data at District 65; and Stacy Beardsley, Assistant Superintendent of Curriculum and Instruction at District 65.
Aligning a GE Score of 8.3 on STAR to a RIT Score of 227 on Map
ETHS gives the STAR test in September to incoming ninth graders, and it has selected a grade equivalent (GE) score of 8.3 on that test to reflect proficiency in reading for an incoming ninth grader. A GE score of 8.3 means the incoming ninth grader has the literacy skills of a “typical” student in the third month of eighth grade, according to information provided by STAR.
Ninth-graders scoring below a GE of 8.3 are “not reading at grade level,” Scott Bramley, Associate Principal for Instruction and Literacy at ETHS, said in a Jan. 10, 2019 memo.
Dr. Bavis told the RoundTable that a GE score of 8.3 was selected to take into account “summer slide” since it is the first assessment given when students return from the summer. He added, “ETHS teachers use STAR data to inform instruction for students, and the scores are also used to indicate the level of support for students who are in Reading Support courses.”
In their joint study, administrators determined that a RIT score of 227 on the reading MAP test given in the Spring of eighth grade reflected a comparable level of proficiency in reading as a GE score of 8.3 on a STAR test given in September of ninth grade.
“There is a high degree of consistency” between the two measures, said Dr. Bavis.
So, the Districts’ joint study found that a RIT score of 227 on the Spring eighth-grade MAP reading test corresponds to a GE score of 8.3 for an incoming ninth grader on STAR.
As it happens, a RIT score of 227 is the same score as the college readiness benchmark score that District 65 has been using for eighth-graders on the Spring MAP test. The score was identified as a college readiness score by the Northwest Evaluation Association, the owner of the MAP test, in a 2015 study.
Thus, a 227 RIT score is not only a college readiness benchmark score, but the joint study conducted by Districts 65 and 202 makes clear that eighth-graders need to achieve that score to be regarded as proficient in reading as they transition to ETHS.
Also, students reading below that level are not reading at grade level from the high school’s perspective.
% Students Proficient in Reading
The report presented data showing the percentages of third- through eighth-graders who were proficient in reading on the Spring 2019 MAP test and the percentages of ninth-, tenth-, and eleventh-graders who were proficient in reading on the Spring STAR tests.
In reporting the percentages of eighth graders who were proficient in reading, District 65 used a RIT score of 227 on the eighth-grade Spring MAP test. The District used scores that were identified by NWEA and the District as college readiness benchmark scores as benchmark scores for proficiency on the third- through seventh-grade Spring MAP tests.
ETHS used GE scores of 9.7, 10.7, and 11.7 to reflect whether a student is proficient in reading at the end of ninth, tenth and eleventh grades. Dr. Bavis told the RoundTable that ETHS chose to use a GE of 9.7 for ninth grade, because “this would suggest that the student was performing similarly to the average student in the ninth grade in the seventh month (April) of the academic year. … We are using the STAR norm of 9.7 because it represents where students should be in April of 9th grade.”
Dr. Bavis added that going from a GE score of 8.3 in September of ninth grade to a GE score of 9.7 in April of ninth grade “expect[s] more than a year’s worth of growth to meet this standard.”
Assuming that is the case, the end-of-year ninth-grade standard used by ETHS may be scaled at a higher level than the end-of-year eighth-grade standard used by District 65, and it may make it look like there is a drop in reading achievement at the high school.
The chart below shows the percentages of black, Latinx, and white third- through eleventh-graders who were proficient on the end of year tests on MAP (for third- through eighth-graders) and STAR (for ninth- through eleventh-graders), using the benchmark scores selected by the Districts.
The data shows that high percentages of students are not proficient in reading, and that there is a wide disparity in the results reported for black and Latinx students and white students.
The data also shows that the percentages of students who were deemed proficient declined across the board at the end of ninth-grade, perhaps because a higher bar for proficiency is being used by ETHS.
Four Grade Levels Behind
Administrators also presented data, by subgroup, showing: 1) the median scores of the group of students who were proficient in reading, and 2) the median scores of the group of students who were not proficient. Again, the results for grades 3-8 are from the MAP tests, and they report median RIT scores. The results for grades 9-11 are from the STAR test, and they report median GE scores. (For Grade 9, the highest GE score reported for STAR is 12.0; for Grade 11, it is 13.0).
The table below reflects the data for the two groups of students. The scores are for the end-of-year tests in 2019.
The scores show there are significant differences between the median scores of these two groups of students. For example, in tenth grade, data provided by the Districts show that 57 black and Hispanic students met the proficiency threshold. The above table shows this group of tenth-graders had a median GE score of 12.6 or higher. Their median score was almost two grades above grade level.
By comparison, there were 183 black and Latinx tenth-graders who did not meet the proficiency threshold, and they had median GE scores of 6.5 and 6.6. Their median score was four grades below grade level.
The difference between the two groups at the tenth grade level is about six grade levels. Some black and Hispanic students are doing very well. Some are behind.
On the positive side, the data shows that the median scores of black, Latinx and white students are going up at each grade level. There is thus growth at each grade level, but not the accelerated growth needed to close the achievement and opportunity gaps.
While the data is not for a matched cohort, the disparities at the end of eleventh grade are similar to those at the end of third grade. The disparities are present at third grade.
The data points out the need to address differences in opportunities and achievement before the third-grade level. This was a key finding made by Sean Reardon, Ph.D., Professor of Poverty and Inequality in Education at Stanford University, in his multi-year study of test data from virtually every K-8 school district across the nation.