A team of academics and researchers headed by Dr. David Figlio of Northwestern University’s Institute for Policy Research will assist School District 202 in its evaluation of the effectiveness of the restructured Freshman Humanities and Freshman Biology courses.

Superintendent Eric Witherspoon, Associate Principal Peter Bavis, Research Director Judith Levinson and Research Associate Carrie Livingston have been meeting on a regular basis since December with Dr. Figlio; Thomas Cook, also of the Institute for Policy Research at Northwestern; Charles Whitaker of Northwestern’s Medill School of Journalism; Lawrence Friedman and Shazia Miller, both of the American Institutes of Research; and John Diamond of Harvard University. Dr. Figlio and Dr. Miller appeared before the District 202 School Board on May 21 to describe the proposed evaluation approach.

“[We have] put together a dynamic technical assistance group,” said Dr. Witherspoon. “They are helping us develop a framework for evaluation which will help us be confident that we are getting the information we need.”

Background

Three years ago, Evanston Township High School restructured its Freshman Humanities course, a team-taught English and history class, to expand opportunities for honors-level credit. This reconfiguration involved consolidating the course into three levels. Students reading below grade-level were placed in a class with intensive literacy-development support. Students testing in the top fifth percentile were placed in an honors-only class. The majority of students were placed in a class that combined regular and honors students. These combined classes were taught the same curriculum by the same teachers as the honors-only class. Students could take this course for regular or honors credit.

This restructured program was designed to allow more students to take honors level courses, and ultimately Advanced Placement (AP) courses.

ETHS expanded the restructuring of the freshman year in 2011. Beginning in the 2011-12 school year, students with reading scores between the 40th and 99th percentiles were placed in the same freshman Humanities classes. ETHS also aligned the curriculum to AP expectations, ACT college-readiness standards, and the Common Core State Standards.

Under this new model, students in English and history Freshman Humanities classes earn honors credit based on the quality of their work throughout the semester. Previously, the honors designation was based on placement criteria that did not take into consideration how students performed in class. The new model requires students to perform well each semester on a series of earned honors-credit benchmark assessments.

ETHS is planning to expand the earned honors-credit restructuring to include biology in 2012-13.

Evaluation Plan

Dr. Figlio introduced the evaluation proposal by acknowledging that, as a parent of two current ETHS students and an incoming freshman, he had some personal interest in the effectiveness of the restructured freshman year.

“I’ve had a lot of opportunity to become familiar with the school and am [glad] to help move this forward,” Dr. Figlio said.

Dr. Figlio explained that the evaluation will focus on the District’s “aspirations” for the restructuring of the freshman year. The overarching goal is “excellence for all and measurably improved outcomes for non-white and economically disadvantaged students.”

More specifically, District aspirations include short- and long-term results such as increased student and parent effort, satisfaction and engagement, increased number of students in higher-level courses throughout their high school career, increased success in higher level courses for a wider range of students, higher ACT scores, increased graduation rates and increased college acceptance at more selective universities.

Dr. Figlio also emphasized that “some outcomes are more easily measured than others. … You cannot just follow one cohort. Instead we need a comparison group. He also pointed out that “students in different cohorts may be fundamentally different.” Finally, he cautioned that “this policy and implementation is a work in progress.”

Given those caveats, Dr. Figlio said five different cohorts of students will be studied: three cohorts of students who were freshmen during the three school years 2008-11 before the current Humanities structure was implemented, one cohort of “Humanities only” freshmen (2011-12) and one cohort of “Humanities + Biology” freshmen (2012-13).

“We have a rich set of background characteristics to control for child differences,” Dr. Figlio said. Some of the data that will be collected includes EXPLORE scores in eighth grade, race/ethnicity, socio-economic status and gender. District 65 is also cooperating by providing “ISAT and MAP trajectories from third through eighth grades,” he added.

The team will look at a range of data to evaluate the effectiveness of the new structure, including attendance, behavior data, ACT scores, AP course selection and performance, graduation rates and college acceptance, attendance and completion.

“It’s important not only to be looking at the average, but how all students over the entire distribution are doing,” said Dr. Figlio.

Although the team of outside consultants has so far been donating their time, Dr. Levinson said, funding will need to be secured in order to proceed with the evaluation on the scope recommended.

“We are currently in the process of seeking funding from the United States Department of Education and the Spencer Foundation,” said Dr. Figlio. “Some outcomes will only be studied if outside funding can be secured. The speed of reporting will depend on availability of funding.”

Board Response

Board members were generally positive about the proposed research and evaluation design.

Gretchen Livingston, who had originally urged administrators to consider consulting with Dr. Figlio, said “It’s great to hear in more detail about what you’ve been working on. I think this will help people understand the approach better and give them some reassurance about whether it’s working or not.”

Ms. Livingston said she was particularly interested in “the college-preparedness and college-success data we can now get from the National Student Clearinghouse. It’s more robust, and we’re going to be better able to track students. … That’s a place we’re going to put focus.”

Dr. Figlio said National Student Clearinghouse data provides information about “college-going” and “persistence” – in other words, data about where students are going and whether they are staying in college. He pointed out that the data was very good on four-year colleges, but not as complete about two-year colleges.

Board member Jonathan Baum questioned the ability of the researchers to “parse out” the effects of any one of the variables that had been changed in the restructuring of the freshman year. He pointed out that the mix of students had been changed as well as the curriculum and the manner in which honors credit is conferred.

Dr. Figlio acknowledged that the situation did present a problem and that “the best that we’re going to be able to do analytically is to think about the package of the three different things going on together.”

Dr. Figlio said that the earliest any reports could be provided would be the end of 2013 and it would be 2018 before data on college attendance and completion could be reported for all the cohorts.