Assessments of Chicago Schools Are Flawed, Report Says

Another result that contradicts state data is a consortium finding of a widening achievement gap, especially during the Duncan administration. Over all, schools in more advantaged areas had stronger gains, while schools in disadvantaged areas tended to show less growth.

September 29, 2011

Assessments of Schools Are Flawed, Report Says

By REBECCA VEVEA

In 20 years of near-constant reform efforts, Chicago’s elementary school students have made few gains, high school students have advanced, and the achievement gap between poor and rich areas has widened, a major University of Chicago study found, contradicting impressions created by years of Chicago Public Schools testing data. 

The report examined performance across three eras of reform over the last two decades — a span including the Argie Johnson, Paul Vallas and Arne Duncan regimes. Researchers for the University of Chicago Consortium on School Research found that most publicly available data measuring the success of public schools in Chicago did not provide an accurate picture of progress. 

One of the most striking findings is that elementary school scores in general remained mostly stagnant, contrary to visible improvement on state exams reported by the Illinois State Board of Education. The consortium study used a complex statistical analysis of data from each state-administered test over the last 20 years, controlling for changes in the test’s content and how it was scored, said Stuart Luppescu, a lead consortium researcher. 

The report supports the argument made by state superintendents, educators and critics of high-stakes testing who say state exams and the requirements of the federal No Child Left Behind Act do not provide an accurate picture of how schools are doing. 

“There are a lot of decisions being made based on that data,” said Elaine Allensworth one of the report’s lead researchers. “If we’re going to be doing that, we’ve got to make sure we’re looking at the tests in a fair way over time.” 

Last week, the Obama administration addressed those concerns by announcing it would allow states to seek waivers from No Child Left Behind rules. Schools that commit to certain reforms including tying teacher pay and performance could gain exemption from a program that seeks to have 100 percent of students meet basic requirements on math and reading scores — commonly called “cut scores” — by 2014. 

“Just using the cut scores can really give you a false sense of change over time,” Ms. Allensworth said. 

Matt Vanover, a state board spokesman, said the department had not yet received the report and could not comment on its findings. 

Starting with data from 1988, when Secretary of Education William Bennett declared Chicago schools the worst in the nation, researchers divided Chicago school reform into three eras based on policies put into effect by the people who led the public school system. 

The first era is characterized by “decentralization” and runs from 1988 to 1995 when Argie Johnson was superintendent. The second era, labeled ”accountability,” began in 1995, after the state turned the school system over to Mayor Richard M. Daley, who appointed Paul Vallas as chief executive. Era 3, “diversification,” covers the Arne Duncan administration from 2002 to 2009. 

Reports from the state board of education showed that the city’s elementary schools were making significant gains, while high schools remained at a standstill. But consortium researchers found that reading scores at the elementary level remained relatively stagnant, while math scores showed only incremental improvement. High schools showed improvements on state scores, but because the bar for improvement is set higher at that level, those gains are not reflected in publicly reported data.

In a statement, the current schools chief, Jean-Claude Brizard, said the findings illustrated the “immediate and critical challenges” the new administration faces and used the lagging results to continue his push for a longer school day. 

Another result that contradicts state data is a consortium finding of a widening achievement gap, especially during the Duncan administration. Over all, schools in more advantaged areas had stronger gains, while schools in disadvantaged areas tended to show less growth. 

The consortium listed eight schools that showed the most significant progress after adjusting for changes in state tests. Seven were located in affluent areas including Lincoln Park, Lakeview, Forest Glen and Norwood Park. 

“One of the disturbing things is the schools most in need of improvement were the least likely to improve,” Ms. Allensworth said. 

Barbara Kent, the principal at Burley Elementary School, which showed the second-highest gains in the study, said her school revamped the reading curriculum and focused on engaging parents, rather than on test preparation. “Really strong instruction leads to strong test scores,” she said. 

The consortium also said that schools that showed growth were strong in the five pillars they had identified as being crucial to success — instructional leadership, adequate professional support, ambitious instruction, a good learning climate, and strong community and family ties. 

In a recent visit to Chicago, Secretary of Education Duncan weighed in on the current reforms taking place in Chicago, applauding Mayor Rahm Emanuel for his push for a longer school day and praising legislators for passing a law that ties teachers’ tenure to performance. 

Of his own reform efforts, Mr. Duncan told The Chicago News Cooperative: “You build upon the past. By no means did we ever then or now declare mission accomplished.”

rvevea@chicagonewscoop.org