Test Monitoring Offers Glimpse Into How City Tries to Preempt Cheating
Each spring, as part of its test monitoring program, the Department of Education disperses a small team to schools on testing days to scrutinize and enforce security guidelines. Some schools are picked randomly, but others were flagged by the department because allegations were lodged by school staff and test score data showed “anomalous” results in recent years, officials said.
During this year’s six-day elementary and middle school testing period in April, education department employees paid 41 visits to 37 schools, according to records obtained by GothamSchools in a Freedom of Information Law request.
The city would not specify which schools were the subject of a targeted monitoring visit, as opposed to a random one. But an analysis of test score data for the schools that had monitors visit showed that many had large increases in 2011, a year when the citywide pass rate barely budged. When monitors visited the schools for the 2012 tests, some of them saw sharp drops on its scores — even while the citywide average increased.
Not all monitored schools saw declines this year and, in fact, some saw large gains. But of the schools that made significant gains on either English or math in 2011, more than half regressed to some degree in 2012. One school’s math proficiency rate dropped by more than 40 percentage points.
The previously undisclosed details about the monitoring program comes at a time when state and federal education officials are increasingly focused on devising policies to improve the integrity of tests in the wake of cheating scandals that have erupted in other cities. The number of schools listed in the monitoring program also provides a limited glimpse into the scope of cheating allegations that the city education department receives and is able to deal with.
(View which schools the city monitored during 2012 tests: www.scribd.com/doc/103573193)
Despite calls for more comprehensive test security measures, the test monitoring program in New York City is shrinking. The number of schools canvassed in 2012 represents about 3 percent of schools that tested students this year, down from about 9 percent last year.
In almost all of the schools, monitors found no evidence of wrongdoing on the days of their visits. They follow a checklist that lists 20 state guidelines to make sure that schools in compliance. Was the test packet’s shrink wrap unmolested and kept in a locked room? Check. Are teachers making sure kids don’t receive inappropriate help? Check. Is the door cracked open and the windows left unobstructed?
But city officials say the program isn’t meant to catch principals or teachers in the act of misconduct. Instead, they say, the program is a successful strategy to stop it from happening in the first place.
“These unannounced visits are effective largely because of their deterrent effect – even schools that don’t actually get a visit they know that they could get a visit at any moment and must demonstrate that their test administration is consistent with regulations,” said department spokeswoman Connie Pankratz.
Officials wouldn’t say if they thought the monitoring presence had a direct effect on how schools performed, but many of the schools did see sharp drops.
At P.S. 270 Johann Dekalb, in Clinton Hill, the percentage of students who passed the English exam dropped 20 points this year, from 63 percent to 43 percent. The school’s math scores fared even worse. The number of students who passed the math dropped from 83 t0 38, which yielded a proficiency rate that was 40 percentage points lower that the year before. .
P.S. 270 was one of four schools that monitors visited twice in April — one day for a portion of the English exam and one day for a portion of the math exam. The three others were Esperanza Preparatory Academy, in Harlem, P.S. 091 The Albany Avenue School, in Crown Heights and Satellite West Middle School, in Fort Greene. Monitors did no visit charter schools in 2012.
Another monitored school, Choir Academy of Harlem, dropped 23 percentage points on math proficiency in 2012, a year after it increased 16 percentage points in 2011. And at P.S. 375 Jackie Robinson School in Brooklyn, the passing rate in math increased 25 points in 2011, then dropped 11 points this year.
Often, the abnormal scoring peaks and valleys didn’t show up when looking at school wide proficiency rates, but did when comparing a grade level from one year to the next.
Continue reading on Gotham Schools…