By Jared Chausow
By Katie Toth
By Elizabeth Flock
By Albert Samaha
By Anna Merlan
By Jon Campbell
By Jon Campbell
By Albert Samaha
Koss offers an explanation for what's happening: Campbell's Law. This principle, first described in 1976 by sociologist Donald Campbell, is probably best summed up this way: The more a test affects decision making, the more likely it will lead to corruption. In particular, wrote Campbell, "when test scores become the goal of the teaching process, they both lose their value as indicators of educational status and distort the educational process in undesirable ways."
As evidence of Campbell's Law in action, Koss points to an audit released by the state comptroller in November, in which a group of independent teachers and education officials were asked to re-score thousands of randomly selected high school exams. On 80 percent of students' papers, the audit found, the re-scorers gave the students a lower grade. Taken in concert with the subsequent National Assessment of Educational Progress (NAEP) math scores—which revealed that, for example, while 85 percent of fourth-graders met state standards, only 35 percent were judged proficient on the NAEP—it was enough to have Koss and other school critics crying foul.
City Department of Education spokesperson David Cantor says that the city would welcome higher state standards, but insists that the disparate test results aren't a problem. "Critics have wanted to say that the NAEP and the state tests are contradictory—they're not," he says. "The reality is the NAEP test is harder. Frankly, the NAEP test tests a significant amount of material that we do not test until higher grades." DOE officials stress that city NAEP math scores have risen between 7 and 11 percent since 2003, which is better than statewide or national trends.
Yet critics say the city's own measures of school success are flawed as well. The Progress Reports are supposed to improve on NCLB's fixed thresholds by crediting schools for year-to-year improvement. But when Columbia Teachers College professor Aaron Pallas and education blogger Jennifer Jennings examined elementary and middle-school improvement scores in the city Progress Reports, they found almost no correlation between which schools scored well year to year. They wrote in the new anthology NYC Schools Under Bloomberg and Klein: "You could actually do better randomly picking schools out of a hat to identify those that would receive high scores for student progress, than by relying on last year's reports as a predictor."
Polakow-Suransky says the DOE plans to tweak the lower-grade Progress Reports to make them more reliable. But Pallas says the city should have seen it coming, since many testing experts advised the city that three-year scores would be a more reliable measure, but that was deemed insufficient for holding principals' feet to the fire on an annual basis. "The problem for the city is they've chosen—especially for the elementary and middle schools—to use measures that they know are going to be unstable from one year to the next."
As for signs that the state tests have been watered down, Pallas says, "I suspect that some people in the DOE know exactly what the numbers are saying. But they have to treat them as legitimate because otherwise they're undermining their own accountability system."
At the same time, many parents and education experts are warning that the effects of high-stakes testing (the term of art for tests where heads roll if scores are low) can cut deep within the day-to-day working of city schools.
First, there are the intimations that schools have been encouraged to cheat to earn higher scores. In 2005, P.S.33 in the Bronx recorded a 50-point jump on the fourth-grade English test, landing its principal a $15,000 bonus under the city's new performance incentives; she promptly retired and, because her pension was based on her final-year salary, earned a $12,000-a-year boost in pension. By 2008, the gains had evaporated, and the school had landed on the Needs Improvement list.
Three years later, former students at P.S.48 in the South Bronx—whose principal, John Hughes, earned a Times profile for overseeing a huge jump in test scores—alleged that teachers had fed them answers to Regents test questions. The DOE subsequently launched an investigation into that school as well as nearby M.S.201, where Hughes had moved on to and which had seen a similar jump in scores. (The DOE says its investigation is still ongoing, and expects to issue findings in the next few weeks.)
Many principals, meanwhile, gripe that the incessant focus on test scores is warping schools' approach to teaching. (As is de rigueur in Klein's DOE, none would talk without being granted anonymity.) One outer-borough principal—who didn't even want his borough named—fumes, "There's no critical thinking. There's no literature. They're telling them what passages to read. That's not education."
Principals also say the pass/fail nature of the AYPs—in which, unlike the city Progress Reports, all that matters is how many kids score a 3 or better, not their overall average—encourage them to devote more time and resources to "bubble kids" who can, with a little help, be bumped up from a 2 to a 3.
A Brooklyn intermediate principal agrees that teaching to the test is rampant, though he insists his school avoids it—mostly. Around one-quarter of his students "hover around the border" between a 2 and a 3, he says. "We make sure those kids, when they're doing their homework and writing assignments, they're being attentive to the things they have to. But we do it through the units they're learning as opposed to just pulling out test prep books."