At the Oct. 10 District 202 School Board meeting, Carrie Levy, Director of Research, Evaluation and Assessment at Evanston Township High School, and Pete Bavis, Assistant Superintendent for Curriculum and Instruction at ETHS, provided the Board with an evaluation of the Restructured Freshman Year Initiative, which was implemented for humanity classes in 2011-12 and expanded to biology in 2012-13.
While the report presents a rather complex analysis, Dr. Bavis provided an overview toward the end of the meeting. The initiative provided access to many students to take honors classes in freshman year who historically would have been foreclosed from doing so, and 381 of them went on to earn honors in freshman year. In so doing, the program created a path for these students to take more rigorous courses in subsequent years.
Students who were in the 50th-69th percentile range as incoming freshman showed significant gains. Students at the top level showed some declines.
Some Background on the Change
Historically, incoming freshmen were placed into a 1 Humanities class (English and history) based on their eighth-grade score in reading on the EXPLORE test, says the Report. There were four placement levels depending on a student’s percentile rank on the test: “regular” (40th – 50th percentiles); “mixed-level-regular” (51st – 69th percentiles); “mixed-level-honors” (70th – 94th percentiles); and “honors” (above the 94th percentile).
As with humanities, there were historically four placement levels for biology, but the percentile ranks defining those levels were somewhat different from those defining the placement levels for humanities.
Under the restructuring, most placement levels were eliminated for 1 Humanities and biology. Incoming freshman scoring above the 40th percentile are all placed into the same 1 Humanities courses. Students scoring above the 50th percentile are all placed in the same biology course. Students meeting certain criteria earn honors credit.
As part of the restructuring program, the curricula for freshman humanities and biology were also changed and aligned to Common Core and ACT College Readiness Standards, Advanced Placement Framework, and Next Generation Science Standards. These changes were intended to make the humanities and biology courses more rigorous than any of the previous courses, including honors-level courses.
Reasons for the Change
As it was being contemplated, the restructuring was the subject of extensive debate and discussion at the Board and community level. Some parents of high-achieving students were concerned that the curriculum in the restructured classes would not be as rigorous as that in the traditional honors classes and that teachers would not be able to differentiate instruction to meet the needs of all students.
A countervailing concern was that most low-income and minority students were being placed in a non-honors track as incoming freshman, based on their results on the EXPLORE or MAP tests. Many thought this placed too much weight on one test and it effectively locked many students out of the opportunity to take honors classes throughout their studies at ETHS.
Dr. Witherspoon said at the time the school “should not be locking kids out of opportunities,” but should “create pathways for many more students to take honors and advanced classes at ETHS and to raise expectations for all students.” The idea was also to provide a more rigorous curriculum to all freshmen taking humanities and biology, including those who scored above the 95th percentile.
“It’s a lot more than three classrooms at the ninth grade,” Dr. Witherspoon said. “We also knew that it was a very important part of a much larger restructuring of a culture of success, a culture of high expectations, a culture where children of all backgrounds, all colors, all socio-economic experiences would have the best opportunities we could have for them.”
One aspect of the restructuring initiative is that it includes an evaluation plan that analyzes how the program is impacting all students, including both those at the higher end and those at the lower end of the achievement spectrum. Dr. David Figlio, a professor at Northwestern University, has assisted in developing the evaluation plan.
Significantly, all of the analyses relate to students who scored above the 40th percentile as incoming freshman. The data is not representative of how all students in the school are doing.
Demonstrating Mastery in the Restructured Courses
On an overall basis, 78% of the freshmen in the restructured program in 2015-16 demonstrated “mastery” in English in at least one semester, compared to 66% of the freshmen in the program in 2012-13. There are similar increases in both History and Biology. Students are deemed to have demonstrated mastery if they have earned both honors credit and a grade of “C” or better in the course.
The accompanying chart, Fig. A, shows the trends by race/ethnicity. It shows that a significantly higher percentage of black and Hispanic students in the program demonstrated mastery in English in 2015-2016 than in 2012-2013. There is a similar trend in History.
In Biology, the percentage of Hispanic students meeting mastery has increased from 41% to 62% during the four-year period. Black and white students showed slight gains.
Only one cohort of students has gone through the freshmen restructuring program for both humanities and English and graduated from ETHS: the 2012-13 freshman cohort (the 2012-13 Cohort). This group only includes students who both scored above the 40th percentile in reading as incoming freshman and who graduated in four-years. It thus includes a relatively higher performing group of students.
The report compares how students in the 2012-13 Cohort did in comparison to a “Comparison Group” using various metrics. The “Comparison Group,” consists of freshman classes in the 2008-09, 2009-10, and 2010-11 school years, all before the restructured program was implemented. So there is a pre-group and a post-group. Data presented for the Comparison Group is the mean for the three-years of classes in that group.
In an effort to assess how the restructured freshman program impacts students at various achievement levels, the report breaks students in the pre- and post-groups into four achievement levels that correspond to the achievement levels historically used to place students. The percentile ranks of these achievement levels are:
• 40 – 50th percentiles,
• 50th – 69th percentiles,
• 70th – 94th percentiles,
• 95th percentile and above.
The accompanying charts, Figures B, C, D, and E, each relate to students who started out in the same achievement level, and they compare how students in the pre- and post-groups performed academically on five metrics. For Example, Chart Fig. B focuses on students who performed in the 40-50th percentile range. The chart shows the percent of students in that percentile range who: 1) scored at/above an 18 on the ACT’s English test, 2) had a composite score of 24 or above on the ACT test, 3) had a composite score of 27 or above on the ACT test, 4) took at least one Advanced Placement (AP) course before graduating, and 5) earned a 3 or higher on at least one Advanced Placement Exam before graduating.
A score of 18 is ACT’s college readiness benchmark for the English test, which corresponds to the 39th national percentile. ETHS did not present data showing the percent of students who met ACT’s benchmarks in reading or science, which correspond to the 56th and the 64th national percentiles, respectively.
A composite score of 22 on the ACT is the composite score indicating college readiness. Students admitted to “more selective” colleges generally have a median composite score of 24 or higher on the ACT. Highly selective colleges generally report that their freshman have a median composite score of 27 or higher on the ACT.
The charts show that the students who started at ETHS in the 50th-69th percentile range showed the highest gains across all metrics. Dr. Levy said all the gains made by that group of students are statistically significant, which she said was a change of 5 percentage points for purposes of this analysis. The only other gain that she said is statistically significant is the increase in the percent of students in the 70th-94th percentile range who had a composite score of 27 on the ACT.
Dr. Levy said students in the top group, those at or above the 95th percentile, showed statistically significant declines in the two metrics: 1) the percent having a composite score or 24 or higher on the ACT, and 2) the percent earning a 3 or higher on an AP Exam.
Board member Mark Metz pointed out that the pre- and post-cohorts were composed of different students, that they had different teachers, different curricula, and that there were a lot of other factors that affected their achievement. “How are we supposed to take this?” he asked.
Dr. Levy said the analysis does not demonstrate causality, but said, “We can look at relationships.” She added that at this point, the report was looking at long-term effects using data about only one cohort of students, and the report represented “early outcomes.” She said when ETHS has data on more cohorts, the report would be “more robust.”
Dr. Bavis noted that the high school’s reports were increasingly looking at how four-year graduates had done, and how students who scored above the 40th percentile as incoming freshman had done. “We need to redouble our look at and focus our efforts to look at kids below the 40th percentile,” he said. “And that’s a growing demographic at ETHS.”
Gretchen Livingston said the City’s Cradle to Career initiative is focusing on that group, and she suggested that ETHS analyze data for students at achievement levels other than those used historically placing students.
Likewise Jonathan Baum said, “The kids below the 40th percentile really have to be the focus in making equity concrete.”
Dr. Bavis said ETHS needed to disaggregate data by deciles (e.g., students performing in the 90th-99th percentile, the 80th-89th percentile, etc.) to really understand what is going on at different achievement levels, and to really evaluate the programs for students scoring below the 40th percentile. He questioned whether the Report on freshman restructuring was relevant.