Date Published: July 3, 2009
[Richard Rothstein is a Research Associate of the Economic Policy Institute. This article is excerpted from his book, Grading Education: Getting Accountability Right (Teachers College Press and the Economic Policy Institute, 2008). For more detail on private sector experience with "pay for performance", see also Teachers, Performance Pay, and Accountability — What Education Should Learn From Other Sectors, by Scott J. Adams, John S. Heywood & Richard Rothstein (Economic Policy Institute, 2009)].
It is conventional to say that holding educators accountable and paying for higher test scores will improve performance. Eli Broad, whose foundation promotes incentive pay for teachers, states, "Virtually every o ther industry compensates employees based on how well they perform. … We know from experience across other industries and sectors that linking performance and pay is a powerful incentive."
Yet in reality, private sector pay is almost never based primarily on quantitative performance measures.
It is not hard to see why. Under No Child Left Behind, reliance on math and reading scores to evaluate performance has corrupted schooling. Educators have responded rationally to incentives that, to devote more time to math and reading, spur reductions in social studies, science, art, music, physical education, cooperative learning and other character-building activities. Reductions have been most severe for disadvantaged students who are most in need of a balanced curriculum. In math and reading themselves, drills leading to limited long-term learning have become commonplace. Some schools manipulate data -- for example, by opportunistic assignment of students to sub-groups where they do the least harm to ratings.
Such corruption could have been foreseen. There are many commonplace illustrations of harm from quantitative accountability.
For example, U.S. News and World Report ranks colleges, based partly on their selectivity - determined by the percentage of admitted applicants (more selective colleges admit a smaller percentage of applicants).
This would be reasonable if the measurement were low stakes. Colleges accepting fewer applicants are likely of higher qu ality, but once this indicator became an accountability measure, colleges had incentives to boost their rejection rates. Some send promotional mailings or drop application fees for unqualified applicants. The acceptance indicator has thus lost much of its value.
Other public sectors have had similar experiences. The government has held local job training agencies accountable for placing unemployed workers in jobs that last at least 90 days. Some agencies then provided child care and transportation to newly hired workers, terminating these services on the 91st day. Other agencies refused to enroll the most difficult-to-place unemployed workers. Others cut back on educational activities designed to train workers for higher-paying and longer-lasting jobs because only short-term employment counted for accountability purposes.
Medicare has issued report cards on health providers. One has been based on mortality rates of open heart surgery patients. Some hospitals and physicians responded by refusing to operate on the sickest patients. Because the accountability system attempted "risk adjustment," statistically controlling for patient characteristics, other providers simply claimed the patients were sicker than they were.
The U.S. General Accounting Office reviewed health care report cards, concluding: "[A]dministrators will place all their organizations' resources in areas that are being measured. Areas that are not highlighted in report cards will be ignored."
Most private-sector jobs, like teaching, include a com posite of easily measured and less-easily measured responsibilities. Because of the ease with which employees game purely quantitative incentives, most private-sector accountability systems blend quantitative and qualitative measures with emphasis on the latter. Certainly, supervisory evaluations of employees may be tainted by favoritism, bias, inflation and even kickbacks or other forms of corruption. That subjective evaluations are so widely used, despite these flaws, suggests that most employers consider quantitative judgment even worse.
Bain and Company, the management consulting firm, advises clients to focus on long-term, not short-term (and more easily quantifiable), goals. A company director estimated that at Bain itself, each manager devotes about 100 hours a year to evaluating five employees for purposes of its incentive pay system. "When I try to imagine a school principal doing 30 reviews, I have trouble," he observed.
Curiously, the federal government administers a balanced approach, simultaneously with its test score-based NCLB. Since 1988, the Commerce Department has made Baldrige National Quality Awards for exemplary businesses. Numerical performance indicators play only a small role. For the private sector, 450 out of 1,000 points are for "results," although even here, some results, such as ethical behavior, social responsibility, trust in senior leadership, workforce capability and customer satisfaction are impossible to quantify.
The Baldrige program was extended to health and education institutions in 1999 . For school districts, 100 of 1,000 points are for student learning outcomes, with other points awarded for subjectively evaluated measures, such as "how senior leaders' personal actions reflect a commitment to the organization's values."
The most recent Baldrige school district award was given in 2005 to the Jenks, Okla., school district. The Commerce Department cited the district's test scores, as well as low teacher turnover and innovative programs, such as an exchange relationship with schools in China and the enlistment of residents of a long-term care facility to mentor kindergartners and pre-kindergartners. Yet in 2006, the Jenks district was deemed by NCLB to be sub-standard because students had failed to make adequate yearly progress in reading test scores.
No comments:
Post a Comment