On Nov. 5, David Wartowski, District 65’s Director of STEM, gave a report on the consolidation of Algebra 8 and Algebra 1 into a mixed-level course called “Algebra for All.” The Board approved the consolidation in June 2017, and at the same time approved “Algebra Excite,” a separate class to provide supports to students.
Mr. Wartowski thanked the 27 teachers who participated in the project and developed professional learning communities to support implementation of the program. Two math coaches were assigned to work with the District’s algebra teachers for the entire 2017-2018 school year.
The District’s STEM department, teachers and coaches agreed that the Measures of Academic Progress (MAP) test would be used to evaluate the impact of the program, but that they wanted to evaluate the program using “a broader span of evidence” than just the MAP test.
Mr. Wartowski presented data showing growth using MAP Conditional Growth Percentiles; a short statement concerning PARCC scores; selected 5Essential survey data; and qualitative student and teacher feedback.
“Rigor has been maintained for the highest level algebra students, and the first year of implementation shows an increase in academic performance among Black students, female students and students in the bottom two quartiles,” said Mr. Wartowski.
While Mr. Wartowski presented data showing student growth, he did not provide any data showing end-of-year status of achievement, such as eighth-graders’ average MAP scores or the percentages of eighth-graders meeting MAP’s college readiness benchmarks.
MAP Conditional Growth Percentiles
Mr. Wartowski said administrators decided to use a measure of growth called “conditional growth percentiles” (CGP) provided by the MAP test to measure the impact of the program.
“We wanted to look at impact and this would be the fairest way to look at impact, because it gives a true apples-to-apples comparison,” said Mr. Wartowski.
The CGP is a student’s percentile rank for growth. MAP compares a student’s growth with that of other students in the nation who were in the same grade and who started out at the same level of achievement.
If a student has a CGP at the 50th percentile, it means the student has growth equal to the average growth of students in the nation who are in the same grade and who started out at the same achievement level. Students who have a CGP higher than the 50th percentile are growing more than the norm. Students who have a CGP lower than the 50th percentile are growing less than the norm.
For example, if a student has a CGP of 66, the student is showing more growth than 65% of the students in the nation who are in the same grade and who started out at the same achievement level.
To measure the impact of consolidating Algebra 8 and Algebra 1, the District compared the CGP of all eighth graders who took Algebra in 2017 with the CGP of all eighth graders who took Algebra in 2018. The comparison was thus not of the same cohort of students. The test used was MAP’s math test, not a subset of questions relating to Algebra. The District used box plots to illustrate the difference between school years.
Mr. Wartowksi presented data broken out by gender, race, quartiles and schools.
By Gender: The District’s Research, Accountability and Data Department (RAD) found that there was a “statistically significant positive increase” in the CGP for female students between 2017 and 2018, said Mr. Wartowski. Male students “saw no significant change from the prior year.”
The RoundTable has estimated the median CGP score for males and females, using a line in the box plots that represents the median score. For males, the median CGP score dropped from about 68 in 2017 to about 66 in 2018. For females, the median score increased from about 58 in 2017 to about 66 in 2018, or by 8 points.
While the increase in 8 points indicates that eighth-grade females are growing more than the norm, District administrators did not quantify what that meant in terms of an increase in a MAP RIT score or fraction of a MAP RIT score.
By Race/Ethnicity: RAD found that there was a “statistically positive increase” in the CGP for Black students between 2017 and 2018, but “all other racial subgroups saw no significant change from the prior year.”
For Black students: The median CGP increased from about 42 in 2017 to 55 in 2018; for Latinx students the median CGP grew from about 52 in 2017 to 58 in 2018; and for White students the median CGP grew from about 71 to 73.
By School: The data by school shows that the major increases in CGP are at Bessie Rhodes which increased from a CGP of about 54 in 2017 to a CGP of about 67, and King Arts, which increased from 52 to 68. Bessie Rhodes and King Arts consolidated Algebra 8 and Algebra 1 in the 2012-2013 school year, so the increase between 2017 and 2018 at those schools was not due to the consolidation of the two programs.
The middle schools – Haven, Nichols and Chute, where the consolidation of Algebra 8 and Algebra 1 took place in 2017-2018 – showed an increase of about 3 to 5 points.
Board member Lindsay Cohen asked if the increase at King Lab could have been due to the supports provided in the Algebra Excite program.
By Quartile: Mr. Wartowski also presented CGP data by quartiles. Students were grouped into the four different quartiles based on their MAP scores as seventh-graders on the Spring 2017 MAP test.
The students’ CGP was measured from the beginning of the eighth-grade year to the end of the eighth-grade year for the school years ending in 2017 and 2018.
The data show that students in the bottom two quartiles showed “a positive significant difference” in CGP scores between 2017 and 2018. The scores for students in the top two quartiles was slightly higher in 2018 than in 2017, but RAD characterized it as not significant.
A chart prepared by RAD is reproduced below. The box for each quartile represents the range of CGP scores for each quartile, and the horizontal line in each box represents the median score. While the box plots illustrate that the largest increases occurred in the bottom two quartiles, the box plots also illustrate the highest growth occurred in the third and fourth quartiles.
For example, students in the bottom quartile had a median CGP of about 30 in 2018, meaning their growth was more than 29% of eighth-graders in the nation. Students in the top quartile had a median growth of about 73, meaning their growth was more than 72% of eighth-graders in the nation.
“One of the concerns that I heard was, ‘Are we going to water down the courses?’ We found out ‘no’ that didn’t happen – no matter how we looked at it,” said Mr. Wartowski. “There was no dip in academic success. The rigor was in fact maintained.”
The above chart that was prepared by School District 65 is a Box and Whisker Plot. The four groupings of box and whisker images present data for four quartiles of District 65 eighth-graders for the school years ending in 2017 and 2018. The students were grouped into quartiles based on their results on the Spring 2017 MAP test in math (the “Quartile Cohorts”), with number 1 being the lowest quartile. The chart compares how much each Quartile Cohort grew in 2017 and 2018, using MAP’s Conditional Growth Percentiles (CGP). Each box with extended “whiskers” at the top and bottom shows the distribution of each Quartile Cohort’s CGP scores in math on the Spring MAP test for the year indicated. The box illustrates the distribution of CGP scores that fell within the middle two quartiles of the Quartile Cohort who took the test; a line which shows up in some of the boxes shows location of the median CGP score. The whiskers extending up from each box shows the distribution of CGP scores that fell within the top quartile of the Quartile Cohort. The whiskers extending down from each box show the distribution of CGP scores that were in the bottom quartile of the Quartile Cohort.
Mr. Wartowski said in a memo to the Board, “An analysis of PARCC scores performed by RAD showed no statistical significance in the difference between students’ math scores between SY 2016-2017 and SY 2017-2018.” The memo, however, did not present any PARCC scores.
The Illinois State Board of Education however, has posted PARCC results on its website. On the 2018 PARCC test, 22% of District 65’s Black eighth-graders, 17% of its Hispanic eighth-graders, and 55% of its White eighth-graders met or exceeded standards in math. The benchmark to meet standards is aligned with college readiness.
The 5Essentials Survey
Mr. Wartowski also presented data from the 2018 5Essentials survey, which is administered under the auspices of the Illinois State Board of Education. The scores are thus norm-based and reflect how a school is doing in terms of implementing each of the 5 Essentials in relation to all other schools in the State that have the same grade configuration. Sixth- through eighth-graders may respond to the survey.
The scoring categories are: a) 0 to 19: least implementation; b) 20 to 39: less implementation; c) 40 to 59: average implementation; d) 60 to 79: more implementation; and e) 80 to 100: most implementation.
The score on Math Instruction was 95, which is very high. Their responses to three subcategories were not as good. The score for Student –Teacher Trust was about 35, an increase from last year; the score for Academic Personalism was about 54, down from last year; the score for Academic Press was about 56, down from last year.
Mr. Wartowski also reported informal student and teacher feedback, which included many comments. His report said 82% of the students agreed “I can learn math.” A low of 24% agreed with the statement, “I like math.”
Teachers indicated as successes, “Students seeing themselves as successful regardless of the class label” and “Rigor, scope, and sequence maintained from Algebra 1.”
Board member Candance Chow said it was important to obtain placement data from Evanston Township High School, so the District would know how many students were required to retake Algebra 1 in freshman year of high school. She pointed out that ETHS says on its website that if students are interested in a STEM field, they need to take geometry in 9th grade.
Ms. Chow also said there is a lack of diversity in the Chem/Phys program and in advanced AP courses at ETHS, and that the majority of the students who take those courses have taken an accelerated math course at District 65. She said it was important for more students to take the accelerated math course at District 65 so they would have access to the higher level courses at ETHS.
Mr. Wartowski said they are exploring “How we can use formative assessments not only from a teacher lens but also to empower students to see where they are in their own learning and make good choices…. That creates independent learners and this is an engine for doing that. … “Recognize where I am in terms of what I know and what I don’t know and what can I do about it.”
Board member Rebeca Mendoza said she was glad the report included feedback from students. She added that the District should help students be their own advocates. “Help students understand what they have to do to get to the next level,” she said.
Pointing to the data showing increased growth for females, Ms. Mendoza said, “It’s a great day to be a girl. That data is really encouraging.”
Board member Sergio Hernandez talked about the need for translation services. “We need to make sure that parents understand the data and understand what they can do to help their kids at home.” He added that he really appreciated going beyond the MAP scores and providing the feedback from students and teachers. He said he would like the format of the report to be the format the District uses in the future to show “how we move kids forward.”
“The data for so long have told us there’s a gap,” said Mr. Hernandez. “The way we’re talking about student success right now is an incredible way to move forward on how we look at all different aspects that impact children and move them from just data points to be these whole human beings that are within a family, within a community, within a school.”
Ms. Cohen said she applauded the work being done, and asked if the District could create Algebra Excite starting at the kindergarten level. “We don’t have to wait for students to get to 8th grade to get these supports that are incredibly useful.”
Mr. Wartowski said, “I do think the model could be carried into any grade level.”
Board Vice President Anya Tanyavutti said, “When we originally talked about the proposal, I felt pretty strongly it was a proposal to put educational equity into action. And the data confirms that.”
Ms. Tanyavutti said, “What we invested in was our belief in our children’s aptitude, and we indicated to our children that we believe you deserve the highest standards of curriculum and instruction, the highest standards of rigor, and we will meet our institutional obligation to give you the supports to access that.
“Part of what I think can be the effect of constantly talking about the gap in opportunity to achieve as an achievement gap is that folks may have lower expectations. Exposure to solutions such as what we’re putting in this proposal and the outcomes that we see is one way to say ‘no,’ keep those expectations high, and here’s a way of doing that.”
She asked, “Do we have our eye on taking the lessons that we learned here with eighth-grade Algebra and applying that to access to Algebra in seventh grade?
Mr. Wartowski said, “Yes.”
Board President Suni Kartha said she loved seeing the hard test data, but also the soft data in terms of feedback from students and teachers. “I would love to have this be the model in how we look at achievement data in all aspects, not just algebra.”
She asked the administration to look at whether there is any predictability about which students take the compacted math class in sixth grade, which allows them to take algebra in seventh grade and geometry in eighth grade.
“I’m cautiously optimistic,” she said. “I think this is showing it’s having the impact we were hoping it would have.”