Ruminations on School Report Card Grades

Susan Crawford to the NYC Education News listserve:

As soon as the DOE began its report cards a few years ago, it seemed clear to me that one of the main functions of the cards was to target which schools could be shut down within two years based on low grades, and then replaced with charter schools or small high schools.

When the DOE was threatened with a lawsuit last spring for trying to shut down schools to replace them with charter schools without approval from District CECs, it backed off. Instead, it has now simply taken to moving in on whatever schools it deems "undercapicity" even if it means taking away classrooms, lunchrooms, and libraries from the "host" schools and leaving them grossly over-crowded, rather than trying to shut them down.

So, from that perspective, the school grades are no longer needed for real estate grabs. But wait,

surely they could be put to some other use! Of course, it's an election year, so make the state tests easier (as they were in 2005), give most schools A's and B's so parents (but more importantly, the non-parent voters out there, who outnumber parents) will think the system has

really gotten turned around. But wait, one problem ... now that all those teachers need bonuses --33% more than budgeted for.

Ms. Forte of DOE says the extra $13 million has found "in central." I couldn't help but wonder if it was found sitting on someone's desk in the form of a check from some large donor whose principal interest this year is in securing another form of real estate -- City Hall.

Susan Crawford

School Report Cards - A Primer

I understand that this what PS 231 in Brooklyn put out to the school community. Lisa

School Report Cards - A Primer November 2007

The school report cards have just been released, and all schools have received a letter grade from A-F. Here are some things you should know about the elementary school report cards and the grading system:



Basics on the Construction of the School Report Card:

* 85% of the report card grade is derived from the state ELA and math test scores: 55% from changes in the test scores from year to year (progress) and 30% from the actual scores (performance).
* 15% of the report card grade is derived from attendance (5%) and parent and teacher surveys (10%).
* Each category - progress, performance, attendance, parent surveys, and teacher surveys - is weighted so that two-thirds of each sub-score is derived from a comparison with 40 peer schools and one-third from a comparison with all city schools.
* A school's peer group is a collection of 40 schools that have a similar population. The way in which similar schools are determined is 40% based on the percentage of African-American and Latino students in a school; 40% based on the percentage of students who qualify for free or reduced lunch; 10% based on percentage of students who receive special education services; and 10% based on the percentage of students who are English Language Learners (ELLs).



Problems with the Construction of the School Report Card:

* The school report card relies too heavily on test scores.
* The DOE has a very restricted definition of progress so that a student who got one question wrong on their New York State ELA test in 4th grade and two wrong on their ELA test in 5th grade is not considered to have made progress, even though the number wrong falls within the standard error of measurement and the student is obviously succeeding at the 5th grade level.
* In elementary schools, the percentage of students in just two out of seven grades who are deemed to have made progress is the single most important factor in determining the school's grade. Third graders' scores only count as part of the 30% of the score that measures performance, and students who are not tested (pre-kindergarten, kindergarten, 1st and 2nd graders) count very minimally in the attendance score which accounts for 5% of the grade.
* The school report card does not take into account the many factors that make a great school, for example, whether a school has enrichments such as music, art, dance, chess and language classes; whether it encourages children to learn to work cooperatively; whether children are making academic progress based on measures other than test scores.
* The grades are based on a curve. The DOE predetermined that this year 15% of schools would score an A (though it actually boosted the percentage to 23%); 40% a B; 30% a C; 10% a D; and 5% an F. There is a very complex formula for determining the grade, and there are minute differences statistically between some schools with different grades.
* Although the DOE has spent a great deal of money on school Quality Reviews, where educators visit the schools and rate them, the results of the Quality Reviews are not factored into the report card grade.
* Because of their reliance on test scores in this report card, the DOE is considering instituting K-2 grades standardized testing, further compounding the problem of an overemphasis on testing and bringing it into grades where it has no place.

Bard President to Meet With City Over C Grade

NY Times

November 9, 2007

By ELISSA GOOTMAN

One of New York’s most prominent educators, Leon Botstein, the president of Bard College, has joined the chorus of criticism over the City Department of Education’s blunt new A through F rating system for public schools, saying in an interview yesterday that it was “reductive” and “depressing.”

But while most city schools received grades this week, Bard High School Early College, associated with Mr. Botstein’s college, did not. Its grade and those of 22 other schools were reported as “under review” by the department. In fact, Mr. Botstein said, he learned last week that the school had earned a tentative grade of C on a draft copy of the report card — even though its graduates earn not just a high school diploma, but two years’ worth of college credits. And he is holding out hope that the grade will be changed.

Mr. Botstein said he sent e-mail messages to Schools Chancellor Joel I. Klein and James Liebman, the department’s chief accountability officer. School officials agreed to meet with Bard officials next week. “I appealed to the chancellor in an effort to tell him to remove this year’s assessment so that a better mode of assessment could be put together,” Mr. Botstein said.

The Bard high school is unique within the city, as the only high school where all graduates leave with a two-year associate degree. Nonetheless, Mr. Botstein’s basic argument is being echoed throughout the city by educators and parents at some schools that, like his, are nontraditional and high-performing. They say that while the new rating system, which is driven by standardized test scores, may be a good way to measure whether schools are imparting basic knowledge, it is less useful and even harmful on the higher end of the performance spectrum.

Mr. Botstein said he respected the chancellor’s need to turn around a failing school system, but urged that he not do it at the expense of innovation and excellence.

“You have a system that is broken and that is failing, and they are desperately trying to improve it. But don’t throw the baby out with the bath water,” he said. “There are a couple of places, and we’re one of them, that really do something different and well.

“Not all plants are weeds,” he said, “so why are you spraying insecticide on the whole thing?”

Mr. Botstein’s appeal to the chancellor was first reported yesterday in The Village Voice online. In an interview from London, he said the criteria used to rate high schools under the new system — based on a complex formula that gives more weight to progress than performance and compares schools with similar populations against one another — seemed particularly out of sync with his school’s mission.

That is because a key factor in students’ scores is state Regents examinations. And at Bard High School Early College, he said, the emphasis is on helping students pass the Regents as quickly as possible — they do so by 10th grade — and by the time they are juniors, they are off to college-level coursework, which he said the city does not take into account.

He said of the Regents, “They’re to a lower standard, and we won’t teach to the test.”

The high school is on the Lower East Side, and Bard plans to open a second one, in Queens, in September.

In a telephone interview, Mr. Liebman said Bard’s grade was not released on Monday because, like the 22 other schools under review, some of its data had not been properly entered. He said the ratings system, which city officials praised as comprehensive and accurate, was equally applicable to schools at all levels of achievement.

“We’ve got to go through the process of analyzing the new data that they’ve supplied that wasn’t filed on time,” he said. “Then we’re going to take a look at that and see what kind of a difference it makes.”

Mr. Botstein has “absolutely not” received any preferential treatment, Mr. Liebman added.

“Just as we have given every other school, we will give Bard the opportunity to convince us that we have made a mistake that in some way puts them on an uneven playing field,” he said. “We want to measure what every school contributes to kids.”

Mr. Liebman said Bard’s grade could well be released before the city’s meeting with Mr. Botstein, saying, “The meeting is coming up very soon, and I don’t know which of those will come first.”

But Mr. Botstein said he would be “very surprised” if the grade came out before the meeting, because the department “acknowledged that we were a different case.”

“They’re in a tough bind, and I have a lot of respect for them,” he said.

“Let’s say we’re a vegetarian restaurant and you’re telling me our meat is not good. I’m telling you we don’t serve meat. We’re not in the meat business.”

Report Cards Spell Closings

BY ELIZABETH GREEN - Staff Reporter of the Sun

November 6, 2007
URL: http://www.nysun.com/article/65895

Principals could be fired and operations shut down altogether at the nearly 150 public schools to which the city gave D and F grades yesterday, and students at the 50 F schools are getting a green light to transfer into a new school as soon as September, school officials said yesterday.

The city schools chancellor, Joel Klein, said school closures, the process where the city phases schools out of existence, dissolving all leadership and teaching positions and sending students elsewhere, could come at a rate unmatched in the last five years. Between 5 and 15 phaseouts have been announced during each year Mayor Bloomberg has held control of the schools.

"Is this a wake-up call for the people that work there? You bet. That's what we're trying to do," Mr. Bloomberg said yesterday, announcing the grades, letters A to F, that are the result of a complicated formula taking into account a school's standardized test scores, graduation rates, and a few other measures.

A careful scouring of information on failing principals — their track record, the number of years they have spent at a school, their school's test score history — is already under way, Mr. Klein said. Meanwhile, as considerations about employment and school closure are mulled, he said schools that received failing grades are also being helped.

The best-ranked schools in the city, those that received As and a high rating on a second qualitative review by outside consultants, are being asked to become "demonstration schools" that can use extra funding to set up sites where they will share ideas and strategies with schools around the city.

Principals are also being urged to log into a new citywide data system, known as ARIS, where they can search out schools that have produced better results with similar kinds of students. For instance, a comparison chart drawn up by school officials highlights schools a TriBeCa middle school, I.S. 289, could look to for ideas on how to raise its D next year. The chart shows 11 other middle schools with high-scoring students that were able to win A's and B's by helping more students, especially low performers, make progress on math and reading tests.

Messrs. Bloomberg and Klein said yesterday that they were prepared for complaints, which they said would come from schools just as they do from students, and they did come quickly.

A parent advocate who has long argued against standardized tests, Jane Hirschmann, stood up during yesterday's press conference to complain about the report cards' heavy emphasis on test scores, which contribute 85% of each grade.

Hours later, the president of the city principals' union, Ernest Logan, sent an e-mail message to principals saying that he is holding off support of the report cards for now. "The ramifications are too great, especially for students in schools that may be mislabeled and the people whose jobs may be on the line," he said.

The president of the teachers' union, Randi Weingarten, also criticized the report cards, saying data in schools is fine — but only as long as it is used to help schools improve, not to punish them.

Behind the complaints was a confusion over how many schools beloved by parents and with solid, longstanding reputations for excellence had received poor marks.

An author of guides to the city's public schools, Clara Hemphill, said she chose not to include the high school that topped the city's ratings, Manhattan Bridges, in her guide of the city's best because of its low attendance rate, 75%. While several schools in Hemphill's guide were rated high, others got poor marks. P.S. 89 in Battery Park City received a C although more than 90% of its students passed a math test last year, and P.S. 3 in Staten Island got an F though more than 98% of students passed the math test.

Ms. Hemphill said parents would be better off relying on their own judgments. "My advice to parents is: Trust your own knowledge," she said. "Staring at a computer screen and trying to figure out what's going on in the school is not all that useful."

Some grades also contradicted state assessments of which schools are failing, or conflicted with quality reports written by outside reviewers last year.

The discrepancies have to do in part with a deliberate twist in how the grades are calculated. While state assessments are based on the number of students who reach proficiency on state exams, the city reports follow what is called a "growth model," with the amount of progress students show from year to year making up 55% of a school's grade. They also focus heavily on whether low performers show gains.

The executive director of a lobbying group, Democrats for Education Reform, Joseph Williams, said parents should take low grades seriously. "It's more productive, instead of being defensive about it, to talk about how you get it to a B and then to an A," he said.

Mr. Williams said the success of the grades will hinge on whether Mr. Bloomberg follows through on the grades with tough consequences.
Mr. Williams said that at his son's Manhattan elementary school, P.S. 11 on 21st Street, a C grade was appropriate — and probably higher than the school would have gotten two years ago, before the arrival of a new principal.

Some principals are already using the report cards as inspiration for change. At the city's elite exam high schools, which were compared against each other in the report cards, principals have begun meeting regularly to share ideas. An early product of their sessions is a conference scheduled for today where teachers will convene for workshops and discussions, the principal at the Bronx High School of Science, Valerie Ready, said.

John Galvin, the assistant principal at a popular Brooklyn middle, I.S. 318, said his school's leadership met to discuss their new grade, a B, but decided not to make any changes. Moving to an A, he said, would require spending many hours on small improvements, moving students who are already passing tests to get just one or two more questions right on a standardized test.

He said test prep would leave students bored, not stronger learners. "We're not going to give up doing art, music, chess, robotics — all the great programs we have during the day that gifted kids are interested in — just to make sure they get a better or equal score than they got the year before," he said. "We do care about the test, but not enough to sacrifice."