The District 65 School Board continued its discussion on setting targets for 37 goals in the five-year strategic plan at its Sept. 9 meeting. The Board’s discussion, however, focused almost entirely on whether the EXPLORE test should be used as one of the mix of tests to assess whether the District was meeting the goal to “Ensure that students graduating from the District have the necessary skills to be successful in high school and adult life.”

School District 202 has given District 65 eighth-graders the EXPLORE test for many years as one of a number of tests to assess incoming freshmen. While District 65 eighth-graders have made substantial gains on the Illinois Standard Achievement Test (ISAT) during the last seven years, they have made little progress on the EXPLORE test in the same period.

Measuring Preparedness for High School and Adult Life

At the Board’s Aug. 3 meeting, several Board members said they wanted to use a mix of tests, rather than just the ISATs, to assess student achievement. Board member Tracy Quattrocki said she thought Board members had reached a consensus to use the ISATs, EXPLORE and the Measures of Academic Progress (MAP) tests.

At the Sept. 9 meeting, Superintendent Hardy Murphy proposed that the District use the ISAT and MAP tests to determine whether students are graduating from District 65 with the skills necessary to be successful in high school and adult life. EXPLORE was not included as part of the mix.

In making that determination, Dr. Murphy said one element considered would be “meeting standards” on the ISATs. He said the District would also look at the number of students performing at or above the 50th and 75th percentile ranks on both the ISAT and MAP tests.

MAP is a computerized-adaptive test developed by the Northern Evaluation Association, and is designed to measure a student’s current instructional level, and may be used to guide differentiated instruction and to measure student growth. The test may be aligned to individual states’ curriculum standards. WILL THE RIT SCALE BE USED OR JUST THE NORM

The District has used MAP in the middle schools for several years to help diagnose how a student is progressing and to adjust or inform instruction for that student. Last year the District expanded MAP to the fifth grade, and this year it is expanding it to the third-grade level. The administration, however, has not previously publicly disclosed MAP test results, and the Board has not used MAP as a measure to assess student achievement on a District-wide basis.

Board member Katie Bailey said, “I’m glad you added MAP. We’re using it here, and I’m glad you’re putting it out there. I think it’s great. I like the 50th and 75th percentile.”

The Debate on EXPLORE

Ms. Quattrocki argued that the District should also use the EXPLORE test to measure student growth.

EXPLORE is part of the ACT family of tests and is designed to be given to eighth- or ninth-graders. The ACT in turn is a part of the Prairie State Achievement (PSAE) test, the state test given to eleventh-graders in Illinois to measure annual yearly progress under the No Child Left Behind Act. The State Board of Education has recently launched a program under which it pays the cost of administering the EXPLORE test to either eighth- or ninth-graders.

 

Paul Brinson, the District’s Director of Information Services, said the EXPLORE test is given at one point in time to eighth-graders, and it is not designed to evaluate the efficacy of eight or nine years of an instructional program. “EXPLORE was designed to predict how well an eighth-grader would do on a college level course,” he said. “It’s to give [students] an idea of how they’re progressing on the likelihood of being successful in college. … It’s not about evaluating a program.”

Judith Levinson, director of research evaluation assessment at ETHS, told the RoundTable that EXPLORE is part of the ACT system, and it is aligned with and correlated with the ACT, which is part of the PSAE. While cautioning that no one test should be used to assess a student’s achievement, she said, “It [EXPLORE] is currently the most predictive measure of how kids will do on the state test [the PSAEs] and the ACT.”

Dr. Levinson added that EXPLORE is a good measure to assess whether a student is prepared for high school and adult life. She said ETHS uses both EXPLORE and MAP to assess incoming freshmen.

A 2008 study, “From High School to the Future: The Pathway to 20” conducted by the Consortium on Chicago School Reseach at the University of Chicago (the U of C Report), says students’ composite scores on EXPLORE say a lot about the likelihood that they will reach a 20 or better on the ACT. The report says a composite score of 20 would give a high school graduate “some moderate chance” of being accepted into Southern Illinois University, Northern Illinois University and University of Illinois at Chicago, where an ACT score of 20 ranked at about the 25th percentile rank for the college class of 2005. The technical manual for the ACT says a benchmark for college readiness is 21.5.

“Virtually no students with very low scores (15 and below) on EXPLORE make it to 20 on the ACT,” says the U of C Report. Data in the report shows that 11 percent of the students who scored 15 and below on EXPLORE went on to score a 20 on the ACT.

According to data released by the School District 202 on Sept. 14, in the 2008-09 school year, the average composite scores for District 65 eighth-graders on the EXPLORE test were as follows: 14.5 for African-American students; 14.8 for Hispanic students; and 19.7 for white students.

Ms. Quattrocki said, “I’ve heard Dr. Murphy say that no one test is a true indicator of a child’s achievement profile. I think the more tests we look at together and the fuller picture we get of how we’re doing as a District, the better – even if we know no one test is perfect and they show different sides of a child’s intellectual capabilities or their learning. I believe we should use what we have, and there is no reason we wouldn’t add that into our profile and into our growth targets.

“It’s so clearly stated in the strategic plan that we want to ensure that students graduating from the District have the necessary skills to be successful in high school and adult life,” she continued. EXPLORE is predictive of how well a child will do on the ACT, which is used for college admission. “If we’re trying to measure how well we’re doing preparing kids for high school, I think we should look at EXPLORE and see if we’re making progress on it,” she said.

“Aim a little higher on EXPLORE every year or evaluate how well we’re doing on it. If we’re trying to find measureable goals, then I think it’s a very obvious tool to measure how well we’re doing in this area.”

 

Ms. Bailey said she was a big proponent of using the MAP scores, which could be used to measure student achievement between third and eighth grade, but said, “I’m not sure about using EXPLORE as an outcome measure.” She suggested using MAP for measurement, as proposed, and asking the adminstration to report annually on the results of the EXPLORE test, which the Board could then review. Ms. Bailey said the issue is “Whether we hold ourselves accountable for increases on EXPLORE.” She said she thought the same thing could be accomplished by using MAP as the measure.

Mr. Brinson said, in his opinion, “If what Ms. Quattrocki is asking for, is there an indicator that’s out there that could somehow satisy the question of preparedness to go on, I think EXPLORE is fine. I’m in agreement with you.” He said, though, that he thought MAP and ISAT give more information about the efficacy of the District’s academic program.

Dr. Murphy said, “When you elevate a test to the level of concern that you’re discussing, it has an impact upon our instructional program, and as a staff that’s what we’re trying not to do. …We feel that MAP will give us enough information.”

“The kind of focus we have had on standards has yielded some success in this District. You may not believe that it’s real success. There’s people who question the validity of that success all over the place. What we’re saying is we’re willing to put MAP into the mix. I’m just saying don’t put so many new tests in the mix that you send the message out there to our teachers and principals that we end up with a fragmented effort, because we’ve got a very focused one at this point, and it has yielded success.”

Assistant Superintendent Susan Schultz said, “If we would say we’re going to focus on the EXPLORE test, then I would instruct Ms. Fogelberg to dismantle the language arts program and go back to teaching grammer exercises and do things differently, because that would inflate our scores [on EXPLORE].”

Ms. Quattrocki responded that the District should not be teaching to the test. “If we’re increasing differentiation [of instruction], … kids would be more prepared for high school and we should be able to measure that through increased perfomance on EXPLORE.”

“The other reason I feel that EXPLORE shouldn’t be ignored,” Ms. Quattrocki said, “is we do see this shrinking of the achievement gap on the ISAT and we haven’t seen it that much on the EXPLORE.” The reason for that should be evaluated, she said.

In the past eight years, minority students have made substantial gains when measured by the meeting standards criteria of the ISATs, but have shown little progress on EXPLORE. For example, 42 percent of the District’s African-American eighth-graders met standards on the 2002 ISATs; the percentage jumped to 84 percent in 2009. By contrast, the average composite scores on the EXPLORE test for District 65’s African-American eighth-graders increased from 14.1 to 14.5 during the same period

Bonnie Lockhart said, “We’re looking at a strategic plan that we have asked the administration to help us determine how it’s going to be done. We’ve asked them to give us the ‘how’ and the ‘what,’ and they’re telling us what they need and what they will use to make this a success. And I think as a Board we need to support that.”

Board president Keith Terry said this issue could be revisited later. “The discussion at this level has begun. We’re beginning to look at this from a number of angles, and we’re no longer where we were. …There is somewhat of a strong argument to looking at different pieces of data so that we can help the various cohorts of kids who struggle throughout this City.”

Mr. Terry said it appeared six Board members did not agree with Ms. Quattrocki about using EXPLORE as a measure, and he moved the discussion to other topics. He said, “This document can be changed over time, as none of us have the right answers right now.”

Does Meeting Standards on ISATs Indicate Being On-track for College Readiness?

District 65 will use “”meeting standards”” on the ISATs as one indicator of preparedness for high school and adult life. According to the Technical Manual for the ISATs, eighth-grade students need a scale score of 245 to meet standards in math on the ISATs.

A 2008 study, “”From High School to the Future: The Pathway to 20″” conducted by the Consortium on Chicago School Research at the University of Chicago, analyzed how students in the Chicago Public School system did on the ISATs and later on the ACTs. The report concluded that eighth-graders who had a scale score of 245 in math on the ISATs have “”virtually no chance of reaching a 20 on the ACT, which we note is an admittedly low bar.”” The report also says, “”The average ACT score for students who ‘meet standards’ [on the ISATs] is 17.5.”” Students who attend a strong high school and earn good grades in high school show higher gains than other students. The report concludes, “”This suggests a major misalignment between our expectation for what students should know and be able to do at the end of elementary school and whether or not they are on track for college readiness.””

On the 2008 ISATs, the average scale scores for District 65 eighth-graders in math were as follows: 262 for African-American students; 266 for Hispanic students; and 303 for white students. The University of Chicago Report goes on to say, “”Math scores of about 265 [which approximates the average scores for District 65 Aftrican-American and Hispanic students] result in about a 20 percent chance of reaching a 20 on the ACT three years later.””