Hello From Melbourne

New York schools have failed the test

Kevin Donnelly

November 12, 2010

Australia's approach to testing students and holding schools and teachers accountable has been copied from New York. The head of schools there, Joel Klein, has just announced his resignation, and an evaluation of the success or otherwise of the New York experience suggests we may be copying his mistakes.
Julia Gillard, when she was education minister, touted New York's system of teacher and school accountability as the world's best. She used the US example to justify Australia-wide testing in literacy and numeracy at years 3, 5, 7 and 9 and making school results public on the My School website.

Advertisement: Story continues below

In 2008 Gillard met Klein in New York and was so impressed with his policies that she invited him to Australia to show his approach of publicly ranking schools and penalising the underperformers.
Klein argues that his reforms have turned around failing schools and raised standards, but the evidence is far from convincing. He refers to improved results in the local New York tests over the past five to six years. But while the local results have improved, based on the more credible National Assessment of Educational Progress tests, the reality is that standards in New York have flatlined.
The suspicion that students achieved better results because the local tests had been made easier to pass was confirmed by a recent independent report, commissioned by the New York Board of Regents and carried out by Professor Daniel Koretz of Harvard University.
When benchmarked against the US national tests, the report says, standards in New York have not improved. Students achieving excellent results in the Klein tests fail to perform in the more academically focused and rigorous Regents examinations.
As Marc Epstein concluded in an analysis published in New York's City Journal, "The feel-good story of rising student test scores over the last several years is largely an illusion produced by dumbed-down tests."
The education scholar Diane Ravitch, who I visited in New York and who is largely responsible for me changing my mind on the benefits of testing and accountability, argues that Klein's approach is "antithetical to good education".
Mirroring Australian critics, Professor Ravitch argues in The Death and Life of the Great American School System that focusing too much on testing the basics leads to a narrow and impoverished curriculum. Such is the pressure on schools to raise standards that subjects such as history, music, physical education and literature decline as teachers give priority to what is being tested.
Given the high-risk nature of testing and accountability, where underperforming schools face eventual closure and teacher pay is linked to performance, Ravitch and other critics say test results are manipulated.
Increasingly, schools are refusing to enrol weaker students, telling parents to keep underperforming students at home on the day of the test. In extreme cases, teachers have been caught cheating by coaching students during a test or providing answers.
Ravitch also argues that Klein's approach, where schools are measured from year to year in terms of how well they improve test results, is flawed and inconsistent. New York schools that consistently perform at the top of the test table are graded D or F (as they show no improvement from year to year) while less academically successful schools are graded A based on the fact that results have improved over time.
A group of testing experts from the National Research Council mirrored many of these concerns in a 2009 letter to the US Secretary of Education, Arne Duncan. They warned about relying too much on standardised, short answer tests such as those introduced in New York, saying "a test score is an estimate rather than an exact measure of what a person knows and can do".
The letter also expresses doubts about the reliability and validity of tests. As the aphorism suggests, there are lies, damned lies and statistics.
As someone who has been a vocal advocate of testing and accountability, I might expect criticism for doing an about-face. But as John Maynard Keynes said, "When the facts change, I change my mind. What do you do, sir?"
Dr Kevin Donnelly is director of the Education Standards Institute and author of Australia's Education Revolution (Connor Court Publishing).