Results of an annual survey designed to measure the effectiveness of the District’s schools show that in the 2014-15 school year School District 65 declined slightly in four of the five areas deemed essential for schools to be effective.
The 5Essentials survey, based on research at the University of Chicago, “provides a comprehensive picture of a school’s organizational culture through student and teacher responses to questions designed to measure five ‘essentials’ critical for school success: effective leaders, collaborative teachers, involved families, supportive environment and ambitious instruction,” Peter Godard, chief officer of Research, Accountability & Data at District 65, said in an Oct. 19 memo.
The 5Essentials are essential elements of an effective school, culled from research at University of Chicago over 20 years in more than 200 schools in Chicago. Their research found that schools that were strong in three or more of these areas “were 10 times more likely to make substantial gains in improving student achievement reading and math than schools that were weak on three or more of the Essentials. Those differences remained true even after controlling for student and school characteristics, including poverty, race, gender, and neighborhood characteristics. Strength on components within the Essentials also correlated with increased teacher retention, student attendance, college enrollment, and high school graduation,” Mr. Godard’s memo said.
In the survey, each school is given an Essential Score for each of the five essentials. The Illinois State Board of Education defines an Essential Score as “a summary indicator that describes the school’s performance on each particular essential.” The scores are reported on a scale of 1-99, where every 20 points is exactly one standard deviation wide, and the benchmark (i.e., the score of 50) is the 2013 Illinois state average by type of school (e.g., K-5, K-8, 6-8 or 9-12).
The scores are thus norm-based and reflect how a school is doing in terms of implementing each of the five essentials in relation to all other schools in the State that have the same grade configuration. The scoring categories are:
• 0 to 19: Least Implementation;
• 20 to 39: Less Implementation;
• 40 to 59: Average Implementation;
• 60 to 79: More Implementation;
• 80 to 100: Most Implementation.
In all but one category – ambitious instruction – the District’s results showed a decline from last year. The score in that category increased from 69 to 72, still keeping it in the “More Implementation” category.
The decline in the other categories between school years 2013-14 and 2014-15 was as follows: effective leaders from 51 to 49; collaborative teachers from 54 to 52; involved families from 66 to 65; and supportive environment from 55 to 51. Even though there were declines in each of these essentials, the District remained at the same implementation level this year in those categories: “Average Implementation” for effective leaders, collaborative teachers and supportive environment and “More Implementation” for involved families.
Mr. Godard’s memo also said there was “excellent participation” in the survey. “Every school had sufficient participation to receive a report, and the response rates were sufficient to allow conclusions about school climate and culture across the District. School-level response rates for the teacher survey ranged from 67 to 99 percent; only four schools had a teacher survey response rate less than 90 percent. The student survey, which was administered only to students in grades six through eight, had school-level response rates ranging from 80 to 89 percent. The parent survey had school-level response rates ranging from 2 to 45 percent; 13 schools had a response rate sufficient to receive a report.”
Changes to the Survey
Because of some changes to the survey, this year’s results may not be comparable to those of the past two years, said Mr. Godard. As examples, he told the Board at the Oct. 19 meeting, two measures in the “effective leaders” category were “revised substantially, so we don’t have an historic trend. … The [new] focus is not just on school principals but on the whole leadership team at the school.”
Mr. Godard also said he advised “using caution in interpreting small changes in scores to indicate a large change in the concept being measured. Rather, I would advise looking at larger changes in score … as evidence of changing conditions in our schools.”
Still, Mr. Godard relied on the results to note the District’s “continued strengths” – involved families and ambitious instruction – and those that need “continued focus” – effective leaders and supportive environments.
For the most part, Board members focused their discussion on specific questions in the survey rather than on the results.
“A lot of this seems to be kind of jargon-ish,” said Claudia Garrison. “What is academic personalization?”
“It has to do with the ways in which a teacher customizes the instruction for the needs of the student,” Mr. Godard replied.
“Differentiation?” Ms. Garrison asked, and Mr. Godard affirmed that.
“What is academic press?” Ms. Garrison asked. Mr. Godard responded, “Academic challenge or rigor.”
“What is collective responsibility?” asked Candance Chow.
Maria Allison, the District’s chief strategy officer, said collective responsibility is “all the teachers in the building taking responsibility for students doing their best, for improving the school, for helping students develop – ‘All kids are our kids. It’s our school.’”
Richard Rykhus said, “The one observation I have is around trust, because that’s a question that’s asked in different relationships. It’s about 46% for teacher-teacher and principal-teacher but for teacher-parent, it goes up 20 points.” He said he could imagine hypotheses to account for the difference – time spent at school, supervisor-employee relationship. “I’ll be interested, when you share these details, what you choose to share and what you observe.”
Tracy Quattrocki asked, “Have you done any analysis to show why there is a 30-point gap between our English and our math instruction [scores] at the middle schools?”
Mr. Godard again urged caution in interpreting those numbers but said the gap between math instruction and English is typical statewide.
“How does that correlate to achievement? … Are you saying that statewide math scores are higher than reading scores? Are you saying that kids are feeling pressed and achieving at a higher rate in math?” Ms. Quattrocki asked.
“I don’t know, off the top of my head,” Mr. Godard said. He also said, “It’s worth a discussion.”
“I think the message to take away here is that when we surveyed last winter, we got results that suggested to us, based on the perspective of teachers and students in grades 6 to 8, that things continued to be in a pretty good place but a place that still needs some improvement in order for us to how increases in test scores and academic achievement,” Mr. Godard said.
Five Essential Aspects of an Effective School
Researchers at University of Chicago found that schools that measured strong in three or more of the five essentials were 10 times more likely to improve student achievement than schools weak in three of more of the essentials. These five essentials, which form the framework of the 5Essentials Survey, are as follows:
• Ambitious instruction: Classes are challenging and engaging.
• Effective leaders: Principals and teachers implement a shared vision for success
• Collaborative teachers: Teachers collaborate to promote professional growth.
• Involved families: The entire staff builds strong external relationships.
• Supportive Environment: The school is safe, demanding and supportive.
The Illinois State Board of Education retained UChicago Impact at the University of Chicago to administer the 5Essentials survey to students in grades 6-12 and to teachers to gather information on how each school and each school district in the State is doing in terms of implementing the five essentials. This is the third year the survey has been given.Organizing the Survey Responses
District 65 officials grouped the questions from the students and teacher portions of the 5Essentials survey “”into a series of survey measures,”” Peter Godard, chief officer of Research, Accountability & Data, said in an Oct. 19 memo.
As an example, he said, the math-instruction measure is “”based on student responses to items like ‘How often are you asked to apply math to situations in life outside of school?’ and ‘How often are you asked to explain how you solved a problem to the class?’
These survey measures, in turn, are organized into scores on the 5Essentials. For example, the ambitious instruction essential is made up of measures of English instruction, math instruction, academic challenge/rigor, and quality of student discussion. The measures and essentials are categorized and scored along the same scale as the 5Essentials rankings.
While the District released the results of its survey this week, it is anticipated that the Illinois State Board of Education will release the 5Essentials data publicly on Oct. 31 as part of the State School Report Cards, said Mr. Godard. This information will be available at www.illinoisreportcard.com.
The 5Essentials form the backbone of the District’s strategic plan, adopted earlier this year. Measures from the 5Essentials are included in the strategic plan strategy indicator scorecard, which the District plans to publish in January, Mr. Godard said.