At the Nov. 7 Policy Committee meeting, District 65 administrators presented “performance indicators” they plan to use to measure progress in meeting the goals of the five-year strategic plan approved by the School Board earlier this year. In terms of academic achievement, the district will measure:
- percentage of students meeting college readiness benchmarks on the Measures of Academic Progress (MAP) test,
- percentage of students meeting standards on the Illinois Assessment of Readiness (IAR), the annual test mandated by the Illinois State Board of Education (ISBE),
- percentage of students scoring below the 25th percentile on the MAP test,
- percentage of students meeting NWEA’s growth targets for the MAP test, and
- percentage of students earning a grade of C or better
While the performance indicators are said to promote equity, district administrators have decided to lower the benchmark for college readiness in reading to the 45th percentile, rather than the 60th percentile that has been used by the district for more than a decade. This reduces expectations for students, and is set significantly below the level that administrators of District 65 and 202 jointly determined is necessary to be a “proficient” reader when entering the high school.
District 65 administrators have also decided to continue using the student growth targets established by NWEA for the MAP test. Those targets expect individual students to stay even with their peers, essentially preserving existing gaps in achievement. They do not set targets that would expect students to make accelerated growth, for example, targets that would expect a student to rise up from the 40th percentile in the third grade to the 60th percentile in eighth grade.
NWEA, the owner of the MAP test, expressly warns that its growth targets do not expect accelerated growth and do not lead to any particular level of proficiency, such as college readiness.
This article summarizes achievement data presented by LaTarsha Green, District 65 Deputy Superintendent, and Simone Griffin, Director of Research, Accountability and Data, at the Finance Committee meeting. In line with the data presented in a Nov. 1 article in the RoundTable, the district’s data shows that students lost ground between 2019 and 2021 but have recouped much of the losses in 2022.
There are still wide gaps in achievement.
The district’s data does not show trends for the period prior to 2019.
Percent of students meeting college readiness benchmarks
One key performance indicator for the strategic plan is the percentage of students meeting benchmarks for college readiness.
While college readiness is still a goal, administrators are lowering the benchmark scores that will be used to measure whether students are on track to college readiness.
In August 2016, the District 65 School Board decided to determine whether students were on track to college readiness by using MAP scores that were linked to the ACT’s benchmarks for college readiness (ACT’s CRBs). ACT’s CRBs predict “B” level work in college. For reading, these MAP scores are, on average, at the 60th national percentile, according to NWEA’s 2020 norm study.
As part of the strategic planning process, administrators at District 65 have decided to determine whether students are on track to college readiness by using MAP scores that are linked to the SAT benchmarks for college readiness (SAT’s CRBs). SAT’s CRBs predict “C” level work in college. For reading, these MAP scores are, on average, at the 45th national percentile, according to NWEA’s norm study.
The new benchmark scores indicating college readiness are significantly lower than the ones adopted by the School Board in 2016. The change will lower District 65’s benchmarks to measure college readiness in reading from the 60th to the 45th percentile.
Taking a look at the eighth-grade benchmark scores can illustrate the impact of the change. At eighth grade, the MAP score linked to ACT’s CRB is 227.1. In contrast, the MAP score linked to SAT’s CRB is 220.2, or 6.9 points less. While a 6.9 point difference may not seem like a lot, the average growth in eighth grade is 4.22 points, according to NWEA’s 2020 norm study. So, a 6.9 difference in MAP scores at eighth grade represents 1.6 years of academic growth.
Importantly, a joint study conducted by administrators of School Districts 65 and 202 determined that an eighth grader at District 65 needed a MAP score of 227 in reading to be viewed as a “proficient” reader when entering freshman year at Evanston Township High School. Students who score below a 227 in reading are slated for interventions.
Despite that, District 65 is reducing its benchmark score for college readiness in reading at the end of eighth grade to a MAP score of 220 – or 1.6 years of growth below the proficiency level needed to succeed at ETHS. Eighth graders who score a 220 in reading on MAP may believe they are on track to college readiness when they graduate from District 65, but when they step into ETHS they may be told they need interventions.
Additional information on this change and the differences in the benchmarks to measure college readiness is provided in the footnote at the end of this article.
College readiness on MAP linked to SAT’s CRB
At the presentation on Nov. 7, Griffin presented a chart showing the percentage of fifth- through eighth-grade students who met college readiness benchmarks on the MAP tests given in the fall of school years 2021-22 and 2022-23.
The chart reflected that in the fall of school year 2022-23, 73% of District 65 students met or exceeded college readiness benchmarks in reading and 45% did so in math. Griffin noted that reading scores remained stagnant while math scores showed improvement in all grade levels with the highest growth in fifth grade.
Because of the width of the chart prepared by the district, it is reprinted below in two sections, one for reading and one for math.
The five-year goal is to increase the percentage of students who meet college readiness in reading from 73% to 76% in five years, and in math from 43% to 46% in five years.
Importantly, though, the benchmarks used to measure college readiness for purposes of the above charts are the MAP scores linked to SAT’s CRBs. For reading, those benchmarks correspond to the 45th percentile.
Students meeting or exceeding standards on IAR
District 65’s performance indicators also include a goal to increase the percentage of students who meet or exceed standards on the IAR. The goal is to increase the percentage in reading from 40.4% (currently) to 55% in five years, and to increase the percentage in math from 40% (currently) to 60% in five years.
The IAR is administered once a year in the spring by ISBE.
On Nov. 7, Griffin presented charts showing the percentage of students who met or exceeded standards on the IAR. ISBE says students who meet or exceed standards in English Language Arts (ELA) and math on the IAR “have demonstrated readiness for the next grade level/course, and, ultimately, are likely on track for college and careers.”
The chart below provides the percentages of students who met or exceeded standards on the IAR in 2019, 2021, and 2022. The charts show a significant drop between 2019 and 2021. Griffin noted there was then “a major increase between 2021 and 2022,” and “we’re getting much closer to our pre-pandemic numbers.” She pointed out that in ELA there was a 3-percentage point difference between 2019 and 2022, and for math there was a 5-percentage point difference from the pre-pandemic number.”
Significantly, the data shows that District 65 is using much lower benchmarks for college readiness on MAP than ISBE is using on the IAR. For example:
- 73% of District 65 students met college readiness benchmarks in reading on the fall MAP test given in the fall of 2022 (using the district’s lowered benchmarks linked to the SAT’s CRBs).
- In contrast, only 41% of District 65 students met or exceeded standards in ELA (reading) on the IAR, which is ISBE’s benchmark for being on track to college and career readiness.
Higher percentages of students will meet a lower benchmark than a higher benchmark.
Griffin also presented charts showing the percentage of students by subgroup who met or exceeded standards on the IAR in 2019, 2021 and 2022. The charts, which are reprinted below, show that between the 2019 IAR test (which was pre-pandemic) and the 2021 IAR test, students in every subgroup (except for students with an IEP in reading) lost ground, but all subgroups rebounded in 2022. Yet, the percentage of students who met or exceeded standards in every subgroup (except students with an IEP in reading) is still slightly lower than in 2019.
It should be noted that only about 45% of the district’s third- through eighth-graders took the IAR in 2021, due to the pandemic. It is unclear if the group of students who took the IAR in 2021 were representative of the universe of students.
Data Management Specialist Karriem Muhammad said that on the 2022 IAR test, various subgroups (e.g., Black students, white students or Latinx students) at five or six schools matched or outperformed their scores on the 2019 IAR test. Thus, he said, some schools were making good progress despite the pandemic.
Superintendent Devon Horton said, “That’s critical. Because when we collectively look at our data as a district and the community, we tend to just see the overall. But there are some phenomenal efforts that are happening in our schools, led by our teachers and our school leaders. … It is critical to know that there was a whole pandemic that has taken place across this world, and our students came back, showing their resilience. Our teachers showed their resilience and their talents.
“We may not be where we would like to be, but we are definitely hopeful about the progress and the direction that our district is going with our work.”
Meeting growth targets
District 65 plans to use NWEA’s growth targets as another key indicator. It plans to increase the percentage of students meeting growth targets in reading from 57% (currently) to 62% in five years, and to increase the percentage meeting growth targets in math from 66% (currently) to 70% in five years. Nationwide, about 50% of the students are expected to meet NWEA’s growth targets.
Under NWEA’s model, an individual student’s growth target is the average growth of students who are in the same grade and who started out the school year at the same achievement level. This means keeping pace with one’s academic peers, staying at the same spot on the distribution scale. Making expected gains does not mean making accelerated growth, i.e. increasing academic and critical thinking skills necessary to move up from, for example, the 40th percentile to the 60th percentile.
If Black and Latinx students have average growth over the next five years, they will be in about the same place as they are now, and the achievement gap will be about the same as it is now. A goal to meet NWEA growth targets may thus reinforce existing achievement gaps. The RoundTable’s analyses in April 2022 and December 2019 covered this in more detail.
Significantly, NWEA’s 2020 norm study expressly acknowledges that it is not designed to provide targets that will accelerate students’ growth to a point where they will be on track to college readiness or meet a certain proficiency standard.
“The norms provide information about average achievement and growth for U.S. students but provide no information about necessary or sufficient achievement or growth in any U.S. state relative to grade-level proficiency or college readiness standards or aspirational growth goals,” according to the NWEA’s Norm Study.
NWEA also said if a school district is interested in accelerating students’ growth or closing achievement gaps, it can set customized accelerated growth targets to achieve those goals, and NWEA’s Norm Study on pages 84-85 provides suggestions on how to do this.
District 65’s strategic plan and its performance indicators have not set customized goals to increase the achievement levels of individual students and to set a path for them to increase year by year to higher levels on the distribution scale.
Breaking out data by race and gender
The district plans to track data by subgroup and in some instances by both race/ethnicity and gender (e.g., Black males, Black females and Black Gender Expansive).
There are significant differences between subgroups when the data is broken out by both race/ethnicity and gender. For example, 36.4% of Black males met college readiness in reading compared to 52% of Black females. The chart below illustrates data provided in the district’s memo.
The “college readiness” data in the above chart is based on the benchmarks for college readiness linked to the SAT’s CRB.
Summary of performance indicators
The district has set additional performance indicators to measure progress in meeting the goals of its five-year strategic plan. The indicators are broken into three groups: Educator Influence, Learning Environment, and Student Learning. Some of these have been discussed above in more detail.
The first group is titled, “Educator Influence.” The performance indicators will monitor the percent FACE Plan implementation; the percentage of staff who have completed at least one Diversity, Equity, Inclusion, Belonging competency training; the percentage of timely and specific feedback to teachers from coaching cycles; the percentage of staff trained in crisis prevention; and the percentage of teacher planning that includes specific student celebrations.
Griffin said the district was making strides in diversifying its staff. She said that 54% of the district’s administrators are white, 42% are Black and 13% are Latinx. She added that 67% of the district’s educators are white, 13% are Black, 11% are Latinx, and 6% are Asian.
The second group of key performance indicators focuses on the Learning Environment. It includes the percentage of K-8 students in attendance; the percentage of students who have one to three discipline referrals; the percentage of suspensions districtwide; and the percentage of suspensions by race, by free- and reduced-fee lunch status, and for students with disabilities.
The Student Learning section contains 10 performance indicators, and for each indicator, there is a “baseline” and a “minimal definitive target. The indicators are listed below. Griffin told the RoundTable the baseline refers to the current student outcomes. If the outcome is for MAP data, it is based on the MAP test given in the fall of 2022. The target represents the goal for the next five years.
- % meeting college readiness benchmarks in reading: baseline 73%, target 76%. (MAP test, using scores linked to SAT CRB)
- % meeting college readiness benchmarks in math: baseline 43%; target 46% %. (MAP test, using scores linked to SAT CRB)
- % at or below the 25th percentile in reading: baseline 16%, target 12% (MAP test)
- % at or below the 25th percentile in math: baseline 19%, target 15% (MAP test)
- % of students meeting/exceeding growth targets in reading: baseline 57%, target 62% (MAP test)
- % of students meeting/exceeding growth targets in math: baseline 66%, target 70% (MAP test)
- % of students meeting and/or exceeding on IAR reading: baseline 40.4%, target 55% (IAR test)
- % of students meeting and/or exceeding on IAR math: baseline 40%, target 60% (IAR test)
- 70% of students with a C or better: baseline 100%, target 70% (C is defined as between 70% and 79%).
- 70% of students with a C of better by race, FRL & students with disabilities: baseline 100%, target 70%
Griffin told the RoundTable that these last two performance indicators relating to grades “represents the goal for all students in all grade levels achieving at least a C or better in their core classes in a marking period.” She added that the district is in the process of shifting to a standards-based grading system, where grades will indicate whether students are exceeding standards, meeting standards, progressing in learning standards, or beginning learning standards.
The district will also create an annual data analysis system that tracks attendance and disciplinary data by both race/ethnicity and gender.
The district plans to publish a scorecard three times a year that will report the percentage of students meeting college readiness benchmarks in reading and math on MAP (using scores linked to SAT’s CRB), the percentage of students scoring below the 25th percentile in reading and math on MAP and the percentage meeting NWEA growth targets in reading and math.
Three members of the School Board attended the Policy Committee meeting: Biz Lindsay-Ryan, Sergio Hernandez and Tracy Olasimbo.
Board member Joey Hailpern submitted a comment before the meeting. He said because the data was not presented by grade level or in smaller cohorts, it was difficult to identify where real growth was occurring and where the real problems were occurring.
Lindsay-Ryan asked if the data could be presented by grade level. Horton responded that administrators and teachers were already analyzing the data by grade level in the field.
Lindsay-Ryan said that good progress was being made at the academic skill centers and asked when that program could be scaled up and expanded. One administrator responded that administrators were thinking about how to refine it, considering it in the context of the budget and other priorities, and “where it makes most sense to focus our attention on this promising model.”
Hernandez said he thought it was promising to see that the district was getting at or near the pre-pandemic outcomes.
He added that as the District 65 and 202 School Boards talk about the joint literacy goal, that “we … look at a broader way and a more comprehensive way of how we measure student outcomes, again, ensuring that students are able to reach college proficiency, or get as close as possible and ensure that we’re closing those gaps.”
“It’s just exciting to see that the growth that we have, that has happened in spite of the pandemic,” added Hernandez. “But, of course, we know that there’s still work to do to make sure that our kids are successful in college and careers. And I know we’ve got a lot of great plans for our middle schools and aligning our work with [District] 202, and that’s what it’s going to take. It’s going take a community-wide collaborative approach to ensure that our students and families are successful, and we normalize success in the school district.”
Footnote re: college readiness benchmarks
The table below reflects two sets of MAP scores for college readiness for fifth through eighth graders. The MAP scores in the second column align with the SAT’s CRBs as determined by the Northwest Evaluation Association in its SAT linking study; and the numbers in the third column are the current percentile ranks of the MAP scores in the second column, as reflected in NWEA’s 2020 norm study.
The MAP scores in the fourth column align with the ACT’s college readiness benchmarks (ACT’s CRBs) as determined by NWEA in it ACT linking study; and the numbers in the fifth column are the percentile ranks of the MAP scores in the fourth column, as reflected in NWEA’s 2020 norm study.
District 65’s adoption of a college reading goal
In August 2011 the District 65 School Board adopted a goal that the district would prepare students to be on track to college readiness and would increase the percentage of students on track to college readiness. District 65 was the first school district in the state to adopt a college readiness goal. One important purpose of the goal was to raise expectations and student achievement for Black and Hispanic students in the district.
The board also decided to measure whether students were on track to college readiness using benchmark scores on the Illinois Standard Achievement Test (ISAT) that were identified by Paul Zavitkovsky, Urban Education Leadership Program at the University of Illinois Chicago. The scores for reading corresponded to the 60th percentile nationally; the scores for math corresponded to the 68th percentile nationally.
At the time, the ISAT was the annual test administered by the Illinois State Board of Education (ISBE) to all third- through eighth graders in Illinois.
The scores identified by Zavitkovsky were linked to the ACT’s college readiness benchmarks, which predict that a student will have a 50% chance of obtaining a B in a related course in freshman year of college. One extensive study at the time showed that 79% of the grades given in four-year colleges were either As or Bs. So a B was a reasonable measure of success.
Adopting benchmarks for MAP
The district decided to measure whether students were on track to college readiness using both the ISAT tests and the MAP test. In August 2016, the District 65 School Board decided to use the MAP scores identified by NWEA in its 2015 study to determine whether students were on track to college readiness. The MAP scores identified by NWEA in that study were linked to the ACT’s college readiness benchmarks. Using NWEA’s 2020 norm study, those MAP scores currently correspond, on average, to the 60th percentile nationally in reading and the 69.5th percentile nationally in math. See the table above.
Last July, Superintendent Horton told the RoundTable that the district was planning to use MAP scores linked to the SAT’s college readiness benchmarks to assess whether students were on track to college readiness. “As far as why the shift from ACT to SAT, I’m sure you are aware that ISBE shifted to the SAT back in 2015,” he said. “This change should have been made back when the state shifted over. It’s really about aligning our assessment outcomes.”
The achievement data and the performance indicators presented at the Nov. 7 Finance Committee meeting use benchmarks for college readiness that are aligned to the SAT’s CRB’s.
The SAT’s CRBs
In 2016, the College Board, the owner of the SAT test, revised the SAT test, and it changed its definition of college readiness. Before then, the College Board defined college readiness as the SAT score at which a student had a 65% chance of obtaining a B- in freshman year college.
In 2016, the College Board changed its benchmark to the SAT score at which a student has a 75% of obtaining a C. This shift significantly lowered the SAT’s benchmark score for college readiness in reading.
In its 2017 study, NWEA identified MAP scores that were linked to the SAT’s CRBs. Those MAP scores currently correspond, on average, to the 44.75th percentile nationally in reading and the 67.5th percentile in math, according to NWEA’s 2020 norm study.
In reading, the change by District 65 from using MAP scores linked to ACT’s CRB to those linked to SAT’s CRB will thus lower the college readiness benchmark for reading from the 60th percentile to the 44.75th percentile. The College Board says that the SAT’s college readiness score for reading corresponds to the 41st national percentile, so the drop may be more than portrayed by NWEA norms.
Of course, the lower the benchmark score, the higher the percentage of students who will meet that score. It will make it look like a School District is doing a better job in educating its students. It may give students an overly optimistic view of how they are doing.
Wealth of data shows significant change
Years of Growth: At eighth grade, the MAP score linked to ACT’s CRB is 227.1; in contrast, the MAP score linked to SAT’s CRB is 220.2, or 6.9 points less. While a 6.9 point difference may not seem like a lot, the average growth in eighth grade is 4.22 points, according to NWEA’s 2020 norm study. So, a 6.9 difference in MAP scores at eighth grade represents 1.6 years of growth (e.g., 6.9/4.22 = 1.635).
Misalignment with IAR: As noted in the text of this article, the Illinois State Board of Education says that a student who meets or exceeds standards on the Illinois Assessment of Readiness (IAR) is likely on track to college readiness. The data shows that the percentage of District 65 students meeting or exceeding state standards in ELA (reading) on the IAR is much lower than the percentage of students meeting college readiness benchmarks on the MAP test (using the District’s lowered benchmarks linked to SAT’s CRBs).
For example, 41% of students met standards in ELA (reading) on the IAR given in the spring of 2022, compared to 73% who were deemed on track to college readiness on the fall MAP test given in 2022 (using the District’s lowered benchmarks linked to the SAT’s CRBs.
The IAR measures to extent to which students are meeting Illinois’ learning standards, what ISBE has determined students should know to be on track to college and career readiness.
Linking to a grade of C: The SAT has linked its college readiness benchmarks to a C in freshman year college. That is a relatively low grade. An extensive study of grading practices at more than 135 four-year colleges with more than 1.5 million students found that 43% of the grades given were As, 33.8% were Bs, 14.9% were Cs, 4.1% were Ds, and 4.2% were Fs. See “Where A Is Ordinary: the Evolution of American College and University Grading 1940-2009.” An update to that study found that in 2013, 79% of the grades were As and Bs, and that the average college student had a GPA of 3.15.
So, 79% of grades given in college are As and Bs. A grade of C is in the bottom quartile. In addition, in most colleges, a GPA of 2.0 is the borderline between passing and failing. A grade of C is just getting by.
Joint Districts 65 and 202 study: In February 2019, administrators of School Districts 65 and 202 agreed in a joint study that an eighth grader at District 65 should obtain a score of 227 on the Spring MAP test to be regarded as “proficient” in reading when entering ninth grade at Evanston Township High School. Students who scored below that level need supports. Administrators confirmed this finding on at least two occasions since 2019. RoundTable articles in 2020 and 2021 go into more detail about the joint study.
Thus, at eighth grade the MAP score indicating proficiency in reading is 227, which happens to be the eighth grade MAP score linked to ACT’s CRB. However, the MAP score now being used by District 65 to indicate college readiness in reading is 220. Students with a score of 220 in reading will be 1.6 year’s worth of growth below where they need to be in order to be considered proficient in reading when they enter ETHS.
ISBE’s View; While the Illinois State Board of Education has decided to administer the SAT test as the State mandated test in high school, it has decided not to use the SAT’s college ready benchmarks in reporting whether students are college ready. ISBE said it chose a more rigorous standard “designed to reduce the likelihood that students would need remedial coursework upon entering college.”
In its 2021 Academic Indicators for a “college and career scholar,” ISBE includes as an indicator ELA, a “Minimum SAT Subject Score of Evidence Based Reading and Writing of 540.” SAT says its college ready score is 480.