Getting your Evanston news from Facebook? Try the Evanston RoundTable’s free daily and weekend email newsletters – sign up now!

As elsewhere in the United States, low-income and minority students at Evanston Township High School are much less likely to take honors and advanced classes than white and high-income students are.

Worried that the District’s policy of relying largely on a single test (the eighth-grade EXPLORE exam) to sort students into different instructional levels in freshman year was contributing to this gap, ETHS has introduced a new model for ninth-grade humanities and biology classes.

Instead of some students automatically receiving honors credit and others being excluded from honors opportunities during freshman year, beginning in 2011-12 most students moved to an “earned honors” model – in which students must perform certain tasks to earn honors – in humanities. This year’s freshmen are also participating in earned honors classes in biology.

Evanstonians have reason to be both optimistic and worried about this change. There’s ample evidence that students who are exposed to challenging coursework achieve greater academic success. But mixed-ability courses have both strengths and weaknesses. While ETHS used existing research to develop its policies, it’s impossible to predict with certainty which students will be made better off by this change – and whether any will suffer as a result.

As a father of three teenagers, two of whom attend ETHS, I was pleased that the ETHS administration and School Board wanted to rigorously examine the benefits and costs of the earned honors initiative. As a scholar who wants to help my community, I was happy to volunteer my time to lead this evaluation. Our team includes a set of experts from Northwestern University, Harvard University and the American Institutes for Research.

In our evaluation, we are comparing two groups of students. The first group entered ETHS before the earned honors initiative took effect. The second group is experiencing the new policies and curriculum. We are following these two sets of kids as they progress through school.

Will they pursue and succeed in advanced placement classes? Will they perform better on the ACT and other important exams? Will they attend more selective colleges and do well there? Will they feel more connected to the school and have higher aspirations for themselves?

We want to see whether the types of kids who have historically done very well continue to do well (or even improve), as well as whether those who have historically done less well perform better or worse under the new system.

There is a lot of uncertainty about what will happen under this new system, and it’s tempting to look to very short-term data, such as freshman grades, for answers about whether the system is succeeding or failing. This is the wrong move.

Simply examining whether certain students are earning good or bad grades under the new system tells us nothing about whether the system is succeeding or failing. In order to judge whether the system is working, we must think about what would have happened if the status quo had been maintained. This is hard to do, and is frequently misunderstood.

Think about the number of people who declared President Obama’s economic policies to be failures when the unemployment rate increased under his watch. What these people failed to consider is what would have happened if the President had done nothing.

Our approach of comparing kids who went through freshman year before and after the policy change helps us deal with this problem. We need outcomes that are comparable across time. Grades in freshman year classes are not comparable, because their meaning changed once the earned honors system was implemented.

But even if we thought we could compare grades in a meaningful way, I would advise against it. It’s always better to evaluate a system on the basis of measures outside of the school’s control. That’s why I want to look at objective benchmarks, such as PLAN scores in 10th grade or ACT scores in 11th grade, success on advanced placement exams and success in college.

So we have to be patient. By the time ETHS receives 10th-grade PLAN scores, prepares a dataset that preserves student confidentiality and gives the evaluators time to analyze the data, it will be at least late 2013. And that’s just for sophomore year outcomes from last year’s freshman class. Add another year before we will see results for the first class that experienced the earned honors system in biology as well as humanities. It won’t be until fall 2014 before we see any junior year results – things like ACT scores, advanced placement performance, and future plans. And only a time-traveler could tell us at that point what will happen to these kids in college.

I’m not saying that ETHS should wait until 2017 or 2018 to decide whether to scrap, maintain, change, or expand the earned honors initiative.

Clearly, if the early indicators this fall or next fall are looking bad, ETHS has an obligation to try something new.

But as an Evanston taxpayer and parent, I want the ETHS administration and school board to make decisions based on evidence – and that means holding our ground for now, paying attention to the evaluation results, and adjusting accordingly when the time comes for action.

David Figlio is the Orrington Lunt Professor of Education and Social Policy and Economics and the Director of the Institute for Policy Research at Northwestern University.