The conflict between a desire for the immediate gratification of short-term data and a commitment to a long-term evaluation by outside experts was evident at the District 202 School Board meeting on June 11, as administrators provided a report about the first year of the earned honors Freshman Humanities course.

The Board approved a biology course last fall with a similar structure to the Freshman Humanities course with the “understanding that a comprehensive evaluation plan would be developed with the assistance of outside evaluators and implemented for all of the redesign of the humanities and biology coursework.”

In her report, Judith Levinson, director of research, evaluation and assessment, reminded the Board that the Technical Advisory Group (headed by Dr. David Figlio of Northwestern University) “has made it very clear that to determine impact we need to look at the effects of the program over students’ high school experience. According to our advisory group, the earliest potential preliminary reporting will be late 2013.”

However, Dr. Levinson and several other administrators did provide the Board with some “formative” information “gleaned from five sources: classroom observations; teacher focus groups and interviews; a student survey; student focus groups; and parent focus groups” and “curricular changes and professional development that will be implemented in humanities for 2012-13 based on [the District’s] experience [with the Humanities course] in 2011-12.

“This (information) was shared with the Technical Advisory Group at our most recent meeting this week and was deemed to be … appropriate … for a first-year formative update,” reported Dr. Levinson.

Dr. Levinson and Carrie Livingston, senior research associate, carried out 83 classroom observations and administered an anonymous student survey to all students in the Freshman Humanities course.

The classroom observations sought to evaluate, among other things, teachers’ use of differentiated instructional strategies and student engagement. The student survey looked at student effort and motivation.

Consultants from the American Institutes for Research, who are also involved in the Technical Advisory Group conducting the long-term evaluation of the freshman restructuring, conducted focus groups with teachers as well as students and parents.

Dr. Levinson cautioned that results of focus groups with students and parents were limited in their value because although 40 students and 90 parents were originally randomly selected, only 10 students and 19 parents ended up participating. Dr. Livingston said the classroom observations were only 30 minutes long rather than the entire 42-minute period, which limited the scope of possible findings.

Dr. Livingston reported that classroom observations indicated that teachers were using a range of instructional techniques with “multiple grouping strategies,” were asking both “explicit” and “inferential” questions, and were checking for evidence of student understanding in a majority of classes. In addition, she also reported that “90 percent of the classroom observations in both English and history found at least 80 percent or more of the students to be on task and participating.”

The student survey, which had a five- point scale, indicated that most students [75 percent] are motivated to do well in Freshman Humanities, at least 65 percent put a lot of effort into their work, both overall and in earning honors credit. Approximately one-third of students found the honors assessments difficult, and about half the students reported contributing to classroom discussions most or all of the time.

In a couple of areas, students seemed somewhat less satisfied. Only 38 percent reported that the Humanities class “makes them think deeply” and fewer than one-third of students strongly agreed with the statement that books and materials for the class “are interesting.”

Several Board members took issue with the fact that there was no information reported about student performance such as grades or earned honors credit.

“I was a little surprised not to see that analysis,” commented Board member Rachel Hayman. “I thought we might get some information about students who would have been placed in regular [-level classes in the past]. Do we have any info that they are rising to the occasion [earning) honors [credit]?”

Board member Gretchen Livingston echoed her remark. “I had assumed that we’d be seeing some information about who’s earning honors credit,” Ms. Livingston said.

“I don’t think that takes away from the review that David Figlio is doing – I don’t think it conflicts with that work,” she added.

Ms. Livingston pointed out that the initial impetus of the change to Freshman Humanities was to “increase the number of students earning honors credit … particularly students of color. … As a school board member, I’d like to see that information.”

Board member Scott Rochelle also suggested that there might be “value” in seeing that kind of information.

Board president Mark Metz reminded his colleagues about the longer term evaluation being conducted by the outside consultants. “We put together a team of renowned experts from around the country – it makes me reticent to not follow their recommendations.” He said that an earlier analysis of such data might be “misinterpreted.”

Board vice president Martha Burns took issue with the request for data about the number of students earning honors credit.

“If the evaluators have said that’s not information we need …. let’s say if there weren’t any kids of color that moved up – does that mean it’s not a good idea [to have restructured the course]?” she asked.

Ms. Burns also pointed out that for the first time, classes are more racially balanced because of the elimination of the straight honors level and that some students are just getting accustomed to those new circumstances.

She suggested that publishing data about grades or honors achievement might have a negative effect on student well-being, a point that Ms. Livingston acknowledged was “well-taken.”

Superintendent Eric Witherspoon urged a long-term perspective.

“I do think it points out how important it is to understand that to be making judgments [at this point is] problematic. [We need to be asking] having taken this more challenging class –what will this mean in the future – if students have this kind of foundation? Whatever we do with the data … we have to be very cautious,” he said.

Mr. Metz appeared to reflect the sentiments of Board members and administrators alike by saying, “One of the things that struck me was the depth and care and diligence that is going into this effort – I feel confident that this may be the most scrutiny in the implementation of [any] new course.”