Sol Stern: New York’s Lake Wobegon Effect

Sol Stern
New York’s Lake Wobegon Effect
The state’s rosy test scores don’t square with reality.
26 June 2008

If New York’s commissioner of education, Richard Mills, is to be believed, one of the great success stories in the history of American public education is unfolding in the Empire State. The commissioner has released 2008 state test results showing that a stunning 97 percent of the 708 third-graders in upstate Warren County are achieving “proficiency” in math. Only five of the county’s third-graders scored at level 1, defined by the test protocol as reflecting “serious academic difficulties.” The state’s other third-graders aren’t doing quite as terrifically as those in Warren County, but they’re pretty close—with 90 percent demonstrating proficiency in math and only 2.3 percent of all children scoring at level 1. For all New York students in grades three through eight, the pass rate in math this year is 81 percent, up 15 percentage points since 2006. Pass rates in reading have also risen dramatically over the past two years—up 7 percent for the state overall; 6.9 percent for New York City; 12.4 percent for Buffalo; 8.2 percent for Rochester; and 8.1 percent for Syracuse.

If these test results, particularly in math, are a true reflection of New York students’ mastery of subject matter, educators should be rushing here from all over the country to learn how our teachers manage to do it. U.S. Secretary of Education Margaret Spellings might also consider visiting the town of Lake George, in the center of Warren County, to congratulate the children there for being the first to achieve the No Child Left Behind Act’s goal of total proficiency—and six years ahead of the target date of 2014, at that. Lake George could then qualify as the real-life version of Garrison Keillor’s fictional Lake Wobegon, where “all the children are above average.”

Alas, New York State’s and Lake George’s test scores are not to be believed. Spectacular improvements of the kind Mills claims just don’t happen within the space of two years for a cohort of over 2 million students in hundreds of school districts with very different demographics. Some extraordinary individual schools do sometimes beat the odds, but nothing like these across-the-board gains has ever been demonstrated in any independent achievement tests, such as the SAT or ACT. Moreover, almost none of the dramatic improvements in the state tests show up in the most recent tests administered by the federal National Assessment of Educational Progress (NAEP), otherwise known as the “nation’s report card.” NAEP scores in fourth- and eighth-grade reading and eighth-grade math in New York State remained flat from 2005 to 2007. There was a modest improvement in fourth-grade math, but nothing close to the gains reported on the state tests. (The next NAEP test is due in 2009.) The inescapable conclusion is that a significant portion of New York’s improvement in test results from 2006 to 2008 was created by the test factor itself—that is, by the degree of rigor (or lack thereof) of the specific state tests administered to the children in those years.

Knowing that last year’s NAEP results sparked some critical commentary (including in the New York Times) about the state tests’ validity, Mills tried to preempt skepticism at his press conference announcing the 2008 results. One of the slides in his PowerPoint presentation was titled ENSURING THESE RESULTS ARE ACCURATE and claimed that “New York’s testing system passed rigorous peer review by [the] U.S. Dep’t of Education.” But this “rigorous peer review,” which all 50 states now undergo under the No Child Left Behind Act (NCLB), is less impressive than it sounds. I was told by a federal education department official that the review covers only the general process used by the states in establishing a reliable system of standards and assessment. It does not constitute a federal seal of approval for the accuracy of any state’s particular tests.

Further, the federal review, though well-intentioned, appears to be producing its own Lake Wobegon effect. The review has already approved 32 states, is about to approve three more, and is still examining the remaining 15 but will eventually approve them, the official told me. Sometime soon, then, the U.S. Department of Education will certify that all 50 states, plus the District of Columbia, have reliable systems for administering the tests that are at the heart of NCLB’s accountability measures. But many state tests will show results that, like New York’s, suggest grade inflation, and there will still be a yawning gap between what many states report about their students’ academic abilities and what the NAEP tests reveal for those states. All of this will produce more fodder for NCLB’s critics on both the left and the right.

The premise of NCLB, as of so many current education reform efforts, is that schools must serve the interests of children, not the interests of the adults who work in the system. But in a classic case of unintended consequences, the widespread test inflation produced by NCLB is serving only the interests of the adults. New York education officials like Mills, New York City mayor Michael Bloomberg, and his schools chancellor, Joel Klein—along with teachers’ union leaders like Randi Weingarten—advance their varied agendas in the glow of inflated test scores. But the children are the big losers. Sometime in the next decade, the white children of Lake George and the black children of New York City will come face to face with reality. On a high school math Regents test—or on an SAT test, or in a college remediation course—they will discover that they are not quite as proficient as New York State once assured them.

Sol Stern is a contributing editor of City Journal and the author of Breaking Free: Public School Lessons and the Imperative of School Choice.