On Aug. 17, District 65 administrators presented a snapshot of student achievement for the 2014-15 school year based on results on the Measures of Academic Progress (MAP) test. The snapshot reports on student achievement using the metrics that the School Board adopted to track progress made under the District’s new five-year strategic plan. Administrators said the District has stopped a downward trend in some areas and made good gains in other areas.
Administrators also presented preliminary recommendations that the Board adopt four goals and growth targets for three of the goals.
The snapshot report does not contain results on the PARCC test. The District is scheduled to provide a more detailed achievement report later this fall.
In January, the District presented an achievement report covering the school years ending in 2011 through 2014. In reading, the data showed noteworthy declines in the percent of students meeting college readiness benchmarks (CRB) and an increase in the percent of students in the bottom quartile. In math, the data showed declines in the percent of students meeting CRB and a significant decline in the percent of students meeting expected gains. Otherwise, the trends were generally flat.
Superintendent Paul Goren said the results were “sobering.” He said the report was a “clarion call” to say, “Let’s build on the foundation that we have, let’s make sure that foundation is strong and then let’s move forward to find progress so that we can make a difference in the lives of kids.”
On March 23, the Board adopted a new five-year strategic plan designed to improve student achievement. Four key metrics that the School Board decided to use to measure progress under the strategic plan are: 1) the percent of students meeting CRB in reading and math; 2) the percent of students in the bottom quartile in reading and math; 3) the percent of students meeting expected gains; and 4) the percent of students, by subgroup, meeting CRB in reading and math.
In reporting on the results on Aug. 17, Peter Goddard, chief officer of research, accountability and data, said, “Overall, district-wide student achievement results on the MAP assessment were relatively positive in 2014-15. The percent meeting college readiness benchmarks and the percent at or below the 25th percentile were flat in both subjects after several years of decreases. Further, we observed a substantial increase in the percent of students making expected gains in both reading and math.”
The table below provides the percentage of students meeting key metrics on the MAP test in 2014-15 and in the prior three years.
Mr. Godard also presented data showing the percent of students, by subgroup, who met CRB in reading and math for the last four years. The data for African American, Hispanic and white students is presented in Charts 1 (reading) and 2 (math). The wide gap in achievement by ethnicity may be due in part to differences in opportunity associated with household income.
These charts show the percentage of District 65 white, Hispanic and black students who were on track to college readiness in the four years indicated, using benchmarks identified by NWEA for the MAP test in a 2011 study. Those benchmarks correspond, on average, to the 82nd percentile in math and the 71st percentile in reading. See sidebar for more information on these benchmarks.
Board member Richard Rykhus asked Mr. Godard to consider, “When we’re seeing some statistically significant change, whether it’s negative or positive, what are the factors that may be driving some of that change?” He asked Mr. Godard to provide three or four reasons when administrators presented the District’s more detailed achievement report in the fall.
Recommended Goals and Targets
Administrators recommended the Board adopt the following four goals:
• Increase the percent of students meeting CRB in math and reading;
• Increase the percent of students making expected gains in math and reading;
• Decrease the percent of students in the bottom quartile in math and reading; and
• Decrease achievement gaps between groups of students in math and reading.
Administrators also recommended that the Board adopt specific five-year growth targets for three of the goals. For the first two goals, administrators recommended that the percentage of students meeting CRB and meeting expected gains increase by 5 percentage points for math and 6.5 percentage points for reading over five years. They recommended that the percentage of students in the bottom quartile be decreased by 3.5 percentage points in math and by 2 percentage points in reading.
“We want to have targets that drive improvement, that are both realistic and aspirational,” said Dr. Goren, “and that are useable and understandable.”
“The whole point of goals and targets is we want to communicate our aspirations and expectations,” said Mr. Godard. He said the targets should take the District’s historical performance into account, recognizing that “change may take time,” and that “lasting improvement may take many years to achieve.”
Mr. Godard said administrators considered setting growth targets by benchmarking against the District’s historical performance, by benchmarking against peers, and by setting aspirational targets. He said benchmarking against peers was not possible because they could not identify comparable districts that used metrics similar to those used by District 65, or access their data. He said it is difficult to ascertain the feasibility of aspirational targets, and added that targets such as “100% of the students” will meet a particular metric can discourage stakeholders if not achieved.
“What you will see tonight in the recommendations I am going to make, we used benchmarking [based on historical performance] to tell us what’s realistic, but then we added a layer of aspiration that is grounded in what we think can really happen with our strategic plan, with our organization as it is today.”
As a first step, Mr. Godard said they set a target level of growth based on the average annual historical growth of the District’s top five performing schools for a particular metric. For example, in the last four years, the average annual growth of the top five performing schools for the metric “% meeting CRB in math,” is 1.4 percentage points. In the model, they assumed it would take time for all schools in the District to get to the level of growth of the top five performing schools.
They thus assumed that growth in math in the first year, 2015-16, would be 0%, then it would be 25% of 1.4 percentage points, then 50% of 1.4 percentage points, then 75% of 1.4 percentage points, and in the fifth year it would be 1.4 percentage points. Adding up the growth in the intervening years, the cumulative increase would be a total of 3.5 percentage points over a five-year period, which is the “modeled growth” for the metric “% meeting CRB in math.” They then added an aspirational amount to identify a “recommended target.”
The table below provides the percentage of students meeting relevant metrics in 2014-15 (the baseline), the modeled targets, and the recommended five-year targets.
While administrators recommended that the Board adopt a goal to close the achievement gap, they did not recommended targets for that goal. Mr. Godard gave a number of reasons for not recommending targets for this goal, including that administrators did not want to set different targets for different subgroups; and the number of students in some subgroups was small and “analytically it’s hard to set a good target,” he said.
Jennifer Phillips said, “I have the highest degree of confidence in what you’re presenting here. It’s really, really impressive.”
“I’ve worked with clients for 20 years doing some of this kind of work, and you’ve knocked my socks off,” said Candance Chow. “What I‘ve talked about with clients is reasoned-based ambitions. So we want our ambition to be reasoned. What we want to happen with this strategic plan is that every teacher in the District looks at these goals and says, ‘yes we can do this.’
“The way you’ve explained the rationale, we are righting a ship,” added Ms. Chow. “We’ve had considerable downward movement over the last five years and to get off that and have some sustainable improvement, that’s not going to happen overnight.”
Richard Rykhus, said, “Kudos. Very impressive work.” He floated the idea of setting the recommended targets as baseline targets and then also setting higher ones that would be more ambitious, something to strive for.
Ms. Chow did not agree with adopting a range of targets, but said, “We should reserve the right next year, after hearing two years of data, to up these targets. … If we’re not aggressive enough and if we have a better track record of upward movement, then we should get more aggressive.”
Board President Tracy Quattrocki suggested that the Board adopt a goal relating to the achievement gap that “there be growth in every subgroup annually. So we wouldn’t set out how much growth, but we would need to see growth in every subgroup.”
Mr. Rykhus said, “I like that.” Other Board members said, “It’s a good amendment,” and “Yes.”
Dr. Goren said, “We want to be explicit that we stand behind the goal of decreasing the achievement gap, full stop. We’ll take that under advisement and see how we can come back.”
Ms. Quattocki asked about a possible change in the benchmark used to measure college readiness (see sidebar) and asked if that were done if it would change the historical profile used in setting the recommended growth targets.
Mr. Godard said if they selected the new CRB, they would redo the analysis and determine whether to adjust the growth targets based on the end-of-year data and the new CRB.
Ms. Quattrocki said, “I think we have to look at this as a living document, because while I think there’s so much integrity to the research, the logic behind it, there is an aspirational quality that is not captured by it, and while we all feel it’s realistic and good, we would like to be shooting a little higher. So I think we need to keep revisiting it and based on our conversation about where we are next year and our conversation about the gap, see if we can adjust those together. That’s my feeling.”
Explanations of ‘CRB’ and ‘Expected Gains’
CRB: District 65 is using college readiness benchmarks (CRB) identified by the Northwest Evaluation Association (NWEA) for the Measures of Academic Progress (MAP) test in a 2011 study. The benchmarks, on average, correspond to the 82nd percentile for math and the 71st percentile for reading. In July 2015, NWEA identified new benchmarks for college readiness that are significantly lower. On average, they correspond to the 68th percentile for math and the 63rd percentile for reading.
Mr. Godard said the CRB used in preparing his snapshot report were based on the 2011 NWEA study. He said the District would consider whether to switch to using the CRB identified in the 2015 NWEA study in conjunction with preparing a joint achievement report with School District 202 this fall.
Paul Zavitkovsky of the Urban Education Leadership Program at the University of Illinois-Chicago, identified benchmarks for college readiness on the Illinois Standard Achievement Test that corresponded to the 68th percentile in math and the 60th percentile in reading.
Expected Gains: To meet expected gains, a student must earn a higher score on the spring MAP test than the average score of students in a national sample who started out school in the same grade with the same MAP score. On a nationwide basis, approximately 50% of the students meet expected gain.
In addition, though, District 65 has added an extra criterion: a student’s gain must be greater than the sum of the standard errors on both the pre- and post-test scores. Mr. Godard said the District added this criterion because “”we don’t want to give ourselves credit for a student having made gains if their gains are statistically the same as neutral or maybe even negative.””