At the May 26 District 202 School Board meeting, Carrie Levy, director of research, evaluation and assessment at Evanston Township High School; Pete Bavis, assistant superintendent for curriculum and instruction at ETHS; and David Figlio, a professor of education and social policy and the director of the Institute for Policy Research at Northwestern University, provided the Board with an evaluation of the Restructured Freshman Year Initiative, which was implemented for humanity classes in 2011-12 and expanded to biology in 2012-13.
While the “early indicators are promising,” the Evaluation Report says, “It is too early in the evaluation to report on any long-term outcomes that will be used to measure the success of the initiative.”
Some Background on the Change
Historically, incoming freshmen were placed into a Humanities class (English and history) based on their eighth-grade score in reading on the EXPLORE test, says the Report. There were four placement levels depending on a student’s percentile rank on the test: “regular” (40th – 50th percentiles); “mixed-level-regular” (51st – 69th percentiles); “mixed-level-honors” (70th – 94th percentiles); and “honors” (above the 94th percentile).
As with humanities, there were historically four placement levels for biology, but the percentile ranks defining those levels were somewhat different from those defining the placement levels for humanities.
Under the restructuring, the placement levels have been eliminated for humanities and biology. Students scoring above the 40th percentile are all placed into the same humanities courses. Students scoring above the 50th percentile are all placed in the same biology course. Students meeting certain criteria earn honors credit.
As part of the restructuring program, the curricula for freshman humanities and biology were also changed. “Unlike the previous courses, the current courses are aligned to Common Core and ACT College Readiness Standards, Advanced Placement Framework, and Next Generation Science Standards,” says the Report. “The courses include more writing, literature analysis, research and skills. All of these changes have made the 1 Humanities and Biology courses more rigorous than any of the previous courses, including honors-level courses.”
Reasons for the Change
The restructuring was the subject of extensive debate and discussion at the Board and community level. Some parents of high-achieving students were concerned that the curriculum in the restructured classes would not be as rigorous as that in the traditional honors classes and that teachers would not be able to differentiate instruction to meet the needs of all students.
A countervailing concern was that most low-income and minority students were being placed in a non-honors track as incoming freshman based on their results on the EXPLORE or MAP tests. Many thought this placed too much weight on one test and it effectively locked many students out of the opportunity to take honors classes throughout their studies at ETHS.
Superintendent Eric Witherspoon said the school “should not be locking kids out of opportunities,” but should “create pathways for many more students to take honors and advanced classes at ETHS and to raise expectations for all students.” The idea was also to provide a more rigorous curriculum to all freshmen taking humanities and biology, including those who scored above the 95th percentile.
“It’s a lot more than three classrooms at the 9th grade,” Dr. Witherspoon said. “We also knew that it was a very important part of a much larger restructuring of a culture of success, a culture of high expectations, a culture where children of all backgrounds, all colors, all socio-economic experiences would have the best opportunities we could have for them.”
One important aspect of the restructuring initiative is that it includes an evaluation plan that analyzes how the program is impacting all students, including both those at the higher end and those at the lower end of the achievement spectrum. The Board asked Dr. Figlio to assist in developing the evaluation plan, and he in turn put together an advisory team composed of seven Ph.D.s from universities around the country to provide input on the best way to do this.
Preliminary Results
In presenting the evaluation Report, Dr. Levy reported on three main areas: “locus of control,” freshman success in earning honors, and enrollment as 11th graders in honors and advanced placement classes.
Locus of Control
Measuring “locus of control” – the extent to which students believe they can control events that affect them – is something new that grabbed the attention of many School Board members. A student survey assessed locus of control by asking students to rate statements such as, “I’m certain I can understand the most difficult material presented in my textbooks,” and “I’m confident I can do an excellent job on my assignments.”
Students who score high believe they have command over their learning. Students who score low believe that external factors have command over their learning. The results show that 39% of all students believe they have a high locus of control; 45%, a medium level of control; and 16%, a low level.
The results indicated that white students are more likely than black or Latino students to believe they have a high locus of control. In English, for example, 46% of white students said they believe they have a high locus of control, compared to 33% of black students and 27% of Latino students.
Dr. Levy said, “It’s one way to learn about students.” If they have a high rating, “they have confidence in their work.” If it is low, they “believe external factors have more control over their learning than they do.”
“We don’t have a lot of context around this yet, and what this quite means,” said Dr. Levy. “So we’re not sure what the findings mean.”
Dr. Bavis said, “What I take it to mean is the sense of agency a student brings to a course: ‘I control my destiny.’ … To me that’s a barometer – something we should move on.”
Locus of control is “one of the things I think is particularly important,” said Dr. Figlio. “Part of the idea behind the freshman humanities program was to change people’s sense of agency, and, ‘Do they have power over their education?’ If you do change that, that’s going to be relatively powerful for me.”
Board member Gretchen Livingston said she was interested to see how the locus of control data broke out by women. “I think it would be very helpful to see that as well,” she said.
Freshmen Earning Honors Credit
The report provides data showing the percent of students in the 2012-13 and the 2013-14 freshman classes who earned honors credit for one semester and for both semesters in English and history. For biology, the report provided data for only the 2013-14 freshman class.
Chart Nos. 1, 2 and 3 graphically present the data for the 2013-14 freshman class, broken out by the four historic placement groups, i.e., those who historically would have been placed in a regular, mixed-regular, mixed-honors, and all honors class.
For example, Chart No. 1 shows:
• 14% of the students who would have historically been assigned to a “regular” level course earned honors credit in both semesters of their English course, and an additional 14% earned honors credit in at least one semester.
• 36% of students who would have historically been assigned to a “mixed-level-regular” class earned honors credit in both semesters, and an additional 20% earned honors credit in at least semester.
• 76% of students who would have historically been assigned to a “mixed-level honors” class earned honors credit in both semesters, and an additional 11% earned honors credit in at least one semester.
• 85% of students who would have historically been assigned to the all “honors” class earned honors credit in both semesters, and an additional 7% earned honors credit in at least one semester.
In order to earn honors credit in freshman year, students must earn a grade of “C” or higher, and also score 80% or better on earned-honors assessments, said Dr. Levy.
Administrators pointed to this data, noting that 28% of the students who would have historically been placed in a regular class and 56% of the students who would have historically been placed in a mixed-level-regular class had earned honors in at least one semester in English. Before the restructuring, these students would not have had that opportunity.
Board member Doug Holt said, “It seems like there’s some real promising early data. Particularly in the middle, with that regular and mixed-honors groups, we’re seeing some expanded opportunities there.”
Mr. Holt asked, though, “Are there any surprises at the low end of the spectrum?” He noted that about 70% of the students who historically would have been placed in a “regular” course did not earn honors credit and asked if that was a concern.
Dr. Witherspoon said, “We never started out with the idea that everyone would earn honors – that’s not the goal. The goal is that everybody would be getting a very rigorous course. … And would that rigor and that strong curriculum give them a base that they could have more pathways, more access once they went beyond the 9th grade?”
The Report does not provide the percentage of students who obtained a D or F in the restructured classes.
Enrollment in Honors/AP as 11th Graders
Dr. Levy said “a primary goal” of the freshman restructuring is to expose students to rigorous coursework so they will have access to and be successful in honors and AP honors courses when they are 11th- and 12th-graders at ETHS.
When the evaluation report was prepared, the 2012-13 freshman cohort had not yet completed 11th grade, so their grades on honors and AP courses were not yet available. The report thus provided the percentage of those students who enrolled in honors and AP courses as 11th-graders in 2014-15.
Chart No. 4 shows the percentage of the 2012-13 freshman cohort who enrolled in one or more honors or AP courses as 11th-graders in 2014-15. The data is broken out by the four historic placement groups, i.e., regular, mixed-level-regular, mixed-level-honors, and honors. The chart also presents data for a “Comparison Group,” which is the mean percentage of students in three prior classes who enrolled in an honors or AP Class.
Chart No. 5 shows the same data as Chart No. 4, except it focuses on enrollment in AP classes (rather than either honors or AP classes).
The charts show that the percentage of students in the 2012-13 freshman cohort who enrolled in honors and AP courses as 11th-graders was the same or higher than that of the Comparison Group. The greatest increase was in the group that would have historically been placed in the mixed-level-regular class, where the increase in enrollement in honors or AP courses jumped from 76% to 89% (Chart 4) and the increase in enrollment in AP courses jumped from 30% to 53% (Chart 5).
Board member Jonathan Baum asked if there was any way to pinpoint how much of the increase in enrollment in AP classes was due to the restructured freshman program and how much was due “to our vigorous efforts to recruit and support new AP enrollment through Team ASAP and AP Boot Camp.”
Dr. Bavis said, “No.”
ETHS’s Report on Student Achievement for 2013-14 reflects that the percent of all 11th- and 12th-graders who took an AP exam at ETHS steadily increased from 45% in 2008-09 to 60% in 2012-13. Progress was thus being made before the first cohort that particpated in the freshman restructuring program reached 11th grade.
Dr. Levy also presented data showing the percentage of students in the Comparison Group and in the 2012-13 freshman cohort who enrolled in one of more AP courses in 11th grade, broken out by race and ethnicity. As summarized in the table below, the data shows that the percentage of black students who enrolled in AP courses increased minimally, and the increase by Hispanic students was the highest.
Too Early to Measure Most Outcomes
“The goal is not just to have more kids taking AP classes,” Ms. Livingston said. “That understates our goal.”
The evaluation plan also includes analyzing scores on AP exams, scores on the ACT test, graduation rates and college acceptance.
Dr. Figlio said he would add to this list where students are going to college. How may are going to selective colleges?
Dr. Levy said it is too early yet to measure these outcomes. The program was implemented for freshman humanities in 2011-12, and for freshman biology in 2012-13. Dr. Figlio cautioned against using data from the implementation year because, “there’s often growing pains with new implementation.” He said he thought “the first decent cohort” to evaluate would be the 2012-13 freshman class, but he appeared to favor using a subsequent cohort to really evaluate the program.
In response to questions, Dr. Figlio said some changes in the format of the Report were based on input of the technical advisory team whose members are conversing regularly to think through the right way to analyze the program. He added the team has collectively devoted between 1,500 and 2,000 uncompensated hours on the project.
The next report will be available in June 2016 and will report for the 2012-13 freshman cohort: 11th-grade course levels and grades; 11th-grade AP scores; 12th-grade course enrollment; ACT scores; ACT survey – college aspirations; and 11-grade locus of control. The report will also report data for the 2013-14 and 2014-15 freshman cohorts.
Dr. Figlio added that he hoped at some point to be able to bring in District 65 scores and to look at trends going back to third grade.
“The data seem to show we’re on the right track,” said Board member Mark Metz. “We’ve got more kids earning honors credit. We’ve got more kids taking this course and doing well, demonstrating mastery. These kids would have been foreclosed and are now given the opportunity and mastering it. … We’ve got some good news.”
“The report is encouraging,” said Board President Pat Savage-Williams.
Dr. Witherspoon said, “Until we get the complete look at this, we can’t jump to conclusions. But the trend lines are showing us things to encourage us.”