Subway Delay Metrics Are Mostly Useless, But Even They Say Service Still Sucks

The numbers don't support Cuomo's "subways are improving" claim


There is at least one person who thinks the subways are getting better: Andrew Cuomo, the man who runs the subways but doesn’t ride them. “I would venture to say if you were looking very carefully, you would see improvement already,” he told reporters on Tuesday from his midtown office. The Daily News added that Cuomo didn’t cite any data to support his claim, but “people have told him it’s gotten better.”

Fortunately, the MTA publishes data about its own performance, so people can investigate just these types of claims. Unfortunately, but not the least bit surprisingly, it is very bad data.

As I’ve written before, the MTA’s performance metrics don’t do a very good job actually telling us how the subway is doing. (MTA chairman Joe Lhota has alluded to new metrics in development on a few occasions but provided no timetable for their deployment.) The MTA uses two numbers to measure performance. The first is on-time performance (OTP), the percentage of trains that make their final stops within five minutes of the MTA’s schedule. (The schedule itself is another issue, not having been updated in decades to reflect operational changes like speed limits.)

According to OTP, Cuomo can claim a technical victory. In June, the last month for which data is available, 61.8 percent of trains reached their last stop on time, up from … 61.7 percent in May. That’s right, a whopping one out of every thousand trains is now more punctual. Congratulations, governor! Mission accomplished!

Or, not so much. OTP was still down 2.3 percent from January and 1.6 percent from April, the unofficial start of the Spring of Suffering. So, if we were to take Cuomo’s claim seriously, improvement over what and when would be the key question. Unfortunately, his anonymous tipsters didn’t clarify.

The other main MTA performance statistic, Wait Assessment (WA), may be the more relevant one to straphangers since waiting is, indeed, the hardest part. Despite the name, WA doesn’t measure waiting all that well, since it still compares arrival time to the heavily flawed schedule, and measures it in a roundabout way. Rather than just measuring how long people wait — like a sensible statistic would — WA instead calculates if the time between trains was less than the scheduled interval, plus a 25 percent cushion. One might argue that waiting 25 percent longer than scheduled is a severe enough delay to count as “waiting” but, alas, the MTA does not agree.

Speaking of things the MTA doesn’t agree with, the agency’s Wait Assessment does not agree with the governor’s assessment that we’re waiting less. WA was down by the equally slim margin of a tenth of a percent from May to June. (In the MTA’s world, a decrease in a stat measuring waiting means you’re waiting more and an increase means you’re waiting less. What do you mean, that’s needlessly confusing?) It’s also no better compared to the winter, when WA was flat from January through April before decreasing one percent from April to May when the Spring of Suffering hit.

The striking thing about all this is not that Cuomo lives in his own fantasy world where the public transportation he never uses is doing better simply because someone told him the tracks look cleaner. The lesson is just how badly the MTA — and by extension, the people who run and control it — measures its own performance. No wonder Cuomo trots out quotes like “if you look closely, you see improvements.” He’s of the mindset that you have to look closely to see the subway is failing New Yorkers in the first place. After all, his own agency is telling him the subway’s performance is only one-tenth of one percent worse than it used to be.

Now, to be fair to Cuomo, the MTA didn’t put forward its subway rehabilitation plan until July, so this data, despite being the most recent available, won’t capture any improvements seen in recent weeks. It’s entirely possible the increased maintenance schedules and super awesome vacuums really are making things better in a quantifiable way. I suppose we’ll just have to take his word for it. He’d never lie to us, right?