Editor’s note: This is one of two analysis pieces from the RoundTable on how Evanston’s school districts fared in the recently released Illinois Report Card, which provides statistical information on school districts throughout the state. Click here for the story on District 202.


District 65’s results on the 2022 Illinois Assessment of Readiness (IAR) show the amount of learning loss during the COVID-19 pandemic. Overall, District 65 students fared better than students statewide.

The data also show that many of District 65’s students, particularly Black and Hispanic students, are not on track to college and career readiness, and that wide opportunity gaps persist. Even aside from the learning loss due to the pandemic, it appears that no progress has been made since 2015, and for white students there has been a downward trend in English Language Arts (ELA).

In addition, there appears to be a trend that significantly lower percentages of eighth graders are on track to college readiness than third graders, suggesting that students may be losing ground after third grade.

Examining achievement data

In presenting achievement reports in prior periods, School District 65 has said, “When aggregate data show that members of a particular student demographic group score below benchmarks, these outcomes reflect opportunity gaps faced by marginalized groups due to institutional racism in the education system and for many families insufficient social and economic supports. These results should not be used to draw conclusions about the efforts, abilities, or strengths of these students or their families.”

In a study published in September 2019, Sean Reardon, Professor of Poverty and Inequality in Education at Stanford University, and his co-authors say, “We examine racial test score gaps because they reflect racial differences in access to educational opportunities.

“By ‘educational opportunities,’ we mean all experiences in a child’s life, from birth onward, that provide opportunities for her to learn, including experiences in children’s homes, childcare settings, neighborhoods, peer groups, and their schools. This implies that test score gaps may result from unequal opportunities either in or out of school; they are not necessarily the result of differences in school quality, resources, or experiences.”

In that spirit, the scores reported in this article measure how well School District 65 and the Evanston community as a whole are providing equitable opportunities to the children in Evanston. [1]

Illinois Assessment of Readiness

The Illinois State Board of Education (ISBE) began administering the IAR in 2019. The test measures how well school districts and communities are doing in preparing students for college and careers. It is Illinois’ annual test administered in compliance with the federal Every Child Succeeds Act. PARCC was the state-mandated test before then. [1]

There are five performance levels for the IAR:
• Level 5: Exceeded expectations
• Level 4: Met expectations
• Level 3: Approached expectations
• Level 2: Partially met expectations
• Level 1: Did not yet meet expectations

“Students performing at levels 4 and 5 met or exceeded expectations, have demonstrated readiness for the next grade level/course, and, ultimately, are likely on track for college and careers,” says ISBE.

In 2019, before the pandemic, the benchmark scores to meet standards on the IAR corresponded to the 62nd percentile in ELA, and to the 68th percentile in math. So, in that year, about 38% of the students in the state met standards in ELA and 32% in math.

Comparing D65 to state results

On the 2022 IAR, 40.5% of School District 65’s students met or exceeded standards on the English language arts (ELA) part of the test, compared with 43.8% on the 2019 IAR, a 3.3 percentage point drop.

For math, 40.1% met or exceeded standards on the 2022 IAR, compared with 45.2% on the 2019 IAR, a 5.1 percentage point drop.

District 65’s learning loss during that period is less than the state averages. On a statewide basis, the percentage of students who met or exceeded standards in ELA declined by 7.7 percentage points in ELA and by 6.3 percentage points in math between the 2019 and 2022.

The chart below compares on an overall basis the declines in ELA and math at District 65 and the declines in the state between 2019 and 2022.

The first sections of this article compares the results on the 2022 IAR with the results on the 2019 IAR, and it gives some indication of the learning loss due to the COVID-19 pandemic.

The 2019 IAR was administered before the outbreak of the pandemic. The next two years are omitted because 1) the IAR was not administered in 2020 due to the pandemic, and 2) District 65 was closed for in-person learning during the 2020-21 school year until Feb. 16, 2021, which is when a limited number of students, on a prioritized basis, were permitted to return for in-person learning for a partial or full day. The 2021 IAR was given to only 46% of the district’s third through eighth graders. It is unclear if that group of students was a representative sample of all of the district’s third through eighth graders.

State schools were open for in-person learning for the entire 2021-22 school year. The 2022 IAR was administered in the spring of 2022, and ISBE released the test results on Oct. 27.

Subsections of this article consider long-term trends, with the data showing that much higher percentages of District 65 students were college-ready in the years 2012-2014. For example, in 2014, 84% of white students and 29% of Black students were on track to college readiness in reading. By 2019, those percentages dropped to 63% for white students and 18% for Black students. On the 2022 IAR, the percentages dropped to 59% for white students and 16% for Black students.

D65 grade 3-8 results

The charts below show the percentage of District 65’s white, Black, Hispanic, Asian and low-income students who met or exceeded standards on the 2019 and 2022 IARs and who are thus viewed as being on track for college and career readiness. The data is taken from ISBE’s website.

The data show two key things:

  • Between 2019 and 2022, there were declines across the board in the percentage of students meeting or exceeding standards. The declines range from two to five percentage points. The pandemic took a toll locally, but not as much as it did statewide.
  • Opportunity gaps persist. The percentages of Black, Hispanic and low-income students who met or exceeded standards in both 2019 and 2022 are well below the percentages for District 65’s white and Asian students.

Performance in three lower levels

The above charts show the percentages of District 65 students who scored in the top two performance levels. The charts below show the percentages of District 65 students who scored in each of the bottom three performance levels in ELA and math on the 2022 IAR. The charts show that very high percentages of Black, Hispanic and low-income students scored in the bottom two performance levels, meaning that they have not yet “approached” meeting learning standards for their grade level.

For example, 66% of Black students fall into the bottom two performance levels in ELA, 39% in the “did not meet” level and 25% in the “partially met” level. Fifty-two percent of Hispanic students fall into the bottom two performance levels. The percentages are similar for math. The data show that high percentages of the District’s students need significant supports.

3rd and 8th grade ELA scores

The chart below shows the percentages of the district’s third and eighth graders who met or exceeded standards in ELA on the 2019 and the 2022 IAR tests. With one exception – Black students on the 2022 IAR – a higher percentage of third graders met standards on the tests than eighth graders.

For example, 69% of white third graders met standards in ELA in 2019, but only 53% of white eighth graders met standards in ELA in 2019. Obviously, this is not a same student cohort analysis, but the pattern (and the long-term trends discussed below) suggest that students are losing ground after third grade.

The data also show that the learning loss between 2019 and 2022 was not uniform by grade level or subgroup.

For example, the data show that the percentage of white students who met standards in ELA dropped from 69% in 2019 to 59% in 2022, or by 10 percentage points.  Black third graders who met standards in ELA also dropped by 10 percentage points between 2019 and 2022; Hispanic third graders dropped by 2 points; and Asian third graders dropped by 12 points.

For math, white eighth-graders dropped 5 points between 2019 and 2022, Hispanic eighth graders dropped 9 points, and Asian eighth graders dropped by 18 points. Black eighth-graders increased by 5 points.

Trends since 2015

The charts below show the percentage of white, Hispanic, Black and low-income students who met or exceeded standards in ELA and math on the PARCC exam for the years 2015 through 2018 and the IAR in 2019, 2021 and 2022. While PARCC and IAR are different tests, IAR is essentially a shorter test, and the scoring is aligned.  

The charts show that wide opportunity gaps between white and Black and Hispanic students persist. (Data for Asian students was not included because the RoundTable lacked data for Asian students for the entire period.)

The top chart shows the percentage of white students who met or exceeded standards in ELA steadily declined between 2015 and 2019, and then dropped significantly in 2021. In 2022, white students recouped some of the losses.

With the exception of white students on the ELA, the trends for the subgroups were similar for both ELA and math. The students’ performance remained generally flat between 2015 and 2019, then dropped significantly in 2021 and then recouped some of the losses in 2022.

In essence, no progress was made between 2015 and 2019, and then there were declines during the pandemic period.

The declines on the 2021 IAR are likely due to the fact that District 65’s schools were closed for in-person learning for all students from March 2020 through Feb. 16, 2021, which is when some students, on a prioritized basis, were permitted to return to school for in-person learning for either a partial or full day. Some students who wanted to return to school for in-person learning were not permitted to because of building capacity and the need for social distancing. A significant portion of the learning loss may be due to the pandemic and the closing of the schools for in-person learning. In addition, though, only 46% of District 65’s third through eighth graders took the 2021 IAR – essentially those who opted to return to in-person learning. It is unknown if the group who took the IAR in that year was a representative sample of the universe of students.

The above charts go back to 2015. Data shows, though, that much higher percentages of District 65 students were college-ready in the years 2012-2014. For example, in 2014, 84% of white students and 29% of Black students were on track to college readiness in ELA. By 2019, those percentages dropped to 63% for white students and 18% for Black students.

In math, 80% of white students and 24% of Black students were on track to college readiness in 2014. By 2014, those percentages dropped to 67% and 14%. [2]

Trends for 3rd and 8th graders

The charts below show the percentage of all third and eighth graders who met or exceeded standards in ELA and math between 2015 and 2022.  The charts show:

  • In pre-COVID years, 2015 to 2019, the percentage of third graders who met or exceeded standards in both ELA and math increased, but there is a sharp decline in 2021 and then a recouping of much of the loss in 2022.
  • For eighth graders, though, there was a relatively steady, significant decline in both ELA and math starting in 2015. In ELA, the percentage of eighth graders who met or exceeded standards has dropped from 49% in 2015 to 32% in 2022. In math, the drop was from 51% to 31%.

Impact of income

Extensive research shows there is a correlation between household income and standardized test scores, and that income levels impact the opportunities that a student has.

 In a 2013 essay titled “No Rich Child Left Behind,” Sean Reardon, Professor of Poverty and Inequality in Education at Stanford University, noted that a household’s degree of wealth and the opportunities it affords may make a significant difference. He says, “The rich now outperform the middle class by as much as the middle class outperform the poor.”

A recent study conducted by a Task Force of the California University Academic Council says, “Enormous differences in income and wealth along lines of race mean that parents have very different resources to invest in their children to prepare them for college. Among these material resources are private schools, tutoring, extracurricular classes, sports teams, educational materials (including test prep), and educational experiences like travel. And the differences in these expenditures have been growing over time, widening the expenditure and opportunity gaps between URMs and others.”

The most recent data available to the RoundTable shows that 70% of Black students attending District 65 in 2017 qualified for free lunch and an additional 6% qualified for reduced-fee lunch, for a combined total of 76%. By comparison, 6% of white students qualified for free or reduced-fee lunch.

The above charts show the correlation between the scores of Black students and students from low-income households.

On Oct. 27, the RoundTable asked District 65 Superintendent Devon Horton if he would like to comment on the district’s results on the 2022 IAR and provided him with copies of four of the charts contained in this article.

Dr. Horton responded, “We just received the report today so we are analyzing our school data. There are entirely new data metrics so we currently do not have any comments. In November we will be presenting our School Report card data comparing 2021 to 2022 as well our Equity progress Indicators for the first quarter of the current school year.”

Footnotes

[1] There are often claims that standardized tests are biased, and historically many researchers have agreed. Whether standardized tests currently being administered are biased is an important question that deserves a close look. On its website, ISBE provides as a resource for the IAR the New Meridian Technical Report 2020–2021 Alternate Blueprint (isbe.net), which explains how the items (or questions) for the IAR were developed. The report, dated Feb. 28, 2022, summarizes the efforts made to remove any bias from the test. It says at pages 15-16:

“Educators and community members make up the committee that reviews items and tasks to confirm that there are no bias or sensitivity issues that would interfere with a student’s ability to achieve his or her best performance. The committee reviewed items and tasks to evaluate adherence to the Fairness and Sensitivity Guidelines, and to ensure that items and tasks do not unfairly advantage or disadvantage one student or group of students over another. Bias and Sensitivity Committee members made edits and modifications to items and passages to eliminate sources of bias and improve accessibility for all students.

“Following the field test, educator and bias committee members met to evaluate test items and associated performance data with regard to appropriateness, level of difficulty, and potential gender, ethnic, or other bias, and then recommended acceptance or rejection of each field-test item for inclusion on an operational assessment. The Data Review Committee also made recommendations that items be revised and re-field tested. Items that were approved by the committee are eligible for use on operational summative assessments.”

In recent years, there has been extensive research of certain standardized tests, particularly college admission tests, to evaluate whether they are biased. Three extensive studies of the SAT concluded that the test did not contain bias against Black or Hispanic students, but rather it overpredicted how Black students would perform in the first year of college. An article about the studies is available here.

In addition, the recent study conducted by the Task Force of California University Academic Council found that “standardized test scores aid in predicting important aspects of student success, including undergraduate grade point average (UGPA), retention, and completion. At [University of California], test scores are currently better predictors of first-year GPA than high school grade point average (HSGPA), and about as good at predicting first-year retention, UGPA, and graduation.” 

The Task Force also found, “In fact, test scores are better predictors of success for students who are Underrepresented Minority students (URMs), who are first-generation, or whose families are low-income.”

The Task Force added, “California high schools vary greatly in grading standards, and that grade inflation is part of why the predictive power of HSGPA has decreased since the last UC study.”

At page 83, the Task Force considered studies about whether there was bias in the SAT and said, “So, in three of four cases (math for both racial/ethnic comparisons and verbal for Latino/white comparisons) no evidence of racial bias emerges. In the fourth case, black/white comparison on the SAT verbal test, some evidence of bias exists, but the bias is against white students on some questions against black students in other cases. Furthermore, our analysis of the results suggest that for this one, the effects are far too small to explain much of the SAT gap in test scores between black and white students.”

Of course, standardized tests at the K-8 grade levels are not used to determine admissions to college. But the point is that standardized tests are predictive, and they serve an important role in assessing whether a school district is educating all of its students at high academic levels and preparing them for college and a career.

In a November 2020 letter sent to the Deputy Assistant Secretary, U.S. Department of Education, a dozen civil rights, social justice, disability rights and education advocacy organizations across the country said, “For the parents, families, and communities we serve, the data from annual statewide assessments are an important source of information that tell them how well the education system is serving their children. The use of standardized tests has helped reveal longstanding achievement gaps and racial disparities in academic opportunity and provided the evidence used by civil rights groups to advocate for change. Systems of accountability in education serve as a critical tool to ensuring the most vulnerable students and schools receive the support and resources they need to succeed.”

[2] In August 2011, the District 65 School Board set as a goal that students have the necessary skills and knowledge to be on track for college readiness. At that time, the Board decided to measure college readiness using benchmarks identified for the Illinois Standard Achievement Test (ISAT) by Paul Zavitkovsky of the Urban Education Leadership Program at the University of Illinois-Chicago. The ISAT was Illinois’ mandated test before PARCC.

The benchmark scores identified by Zavitkovsky for the ISAT corresponded to the 60th Illinois percentile in reading and the 68th Illinois percentile in math. The scores were predictive of having a 50% chance of obtaining a B in a related course in the first year of college. 

On an overall basis, the benchmark scores to meet or exceed standards on the PARCC and the 2019 IAR correspond to the 62nd Illinois percentile in reading and the 68th Illinois percentile in math – virtually the same as those identified by Zavitkovsky for the ISAT.

While the percentile ranks for college and career readiness match up for the ISAT, PARCC and 2019 IAR tests, the charts below show that significantly lower percentages of District 65 students met the benchmarks for college readiness on the 2017 and 2018 PARCC and the 2019 IAR tests, than on the 2012, 2013 and 2014 ISATs.

For example:

  • On the 2014 ISAT, 84% of white students met the college readiness benchmark in reading; on the 2019 IAR, only 63% did.
  • On the 2014 ISAT, 29% of Black students met the college readiness benchmark in ELA; on the 2019 IAR, only 18% did.

The data for the ISATs in the charts below was taken from District 65’s achievement reports. The data for PARCC and IAR was taken from ISBE’s website.

For more data on this, click here.

Larry Gavin

Larry Gavin was a co-founder of the Evanston RoundTable in 1998 and assisted in its conversion to a non-profit in 2021. He has received many journalism awards for his articles on education, housing and...

Join the Conversation

3 Comments

The RoundTable will try to post comments within a few hours, but there may be a longer delay at times. Comments containing mean-spirited, libelous or ad hominem attacks will not be posted. Your full name and email is required. We do not post anonymous comments. Your e-mail will not be posted.

Your email address will not be published. Required fields are marked *

  1. If you examine the data in the first chart it is evident that there was a decline in performance from 2015 to 2019 before the pandemic. Does anyone have an explanation for that? Also it would be interesting to see data going back further in time. I wonder if that could be made available in a future article.

  2. Part of the challenge in comparing 2019 to the present is that, due to -1300 student enrollment you’re comparing apples to oranges. The students that left were those with the resources to send their children to private schools.

    Given that, this drop actually seems pretty not-so-bad and we should commend our teachers for doing the best they have with the limited support they’ve gotten from the administration and board.

    With respect to the opportunity gaps, I think the community needs to re-assess what we can do to close that gap because the current approach (pouring money into equity consultants and expensive coaches) doesn’t appear to be moving the needle very much. At what point do we say, let’s try something else? More than 1% of the D65 budget has gone to this cause and the numbers have only gotten worse! Perhaps the expensive leadership coaches aren’t helping.