NYC's Computer-System Cash-Dump Disaster
Photo-Illustration by Andrea Levy
In January 2009, a tantalizing and disturbing comment materialized on an Internet bulletin board about an expensive and controversial Bloomberg administration project to automate the city's payroll system, known as CityTime.
The anonymous author alleged that the project was hopelessly corrupt and out of control and had been for years. The writer, who claimed to have been employed on the project, went on to name three people he alleged were responsible for that corruption.
The commenter accused a consultant, Mark Mazer, of being "the most crooked person on the team," and said consultant Scott Berger was building a home in Florida at city expense.
"Mark Mazer and Scott had ONLY one main intent . . . to pocket the $/hr for themselves for as long as possible @ taxpayer expense," the commenter wrote. Referring to the former consultant later appointed to oversee the project, the commenter added, "The project in its 5th year was a failure and should have been canned, but Joel Bondy for some reason or another decided it must go on for another 5 yrs."
The comment went on to lay out in more or less clear terms exactly what was taking place in the CityTime project.
That posting appears to have disappeared into the depths of the Internet, but it turned out to be prescient.
Berger, who worked for a CityTime consultant called Spherion, Mazer, and four other people were indicted last month for defrauding the city of $80 million—a theft that made the Mafia's $6 million Lufthansa heist in 1978 look like a bodega stickup job.
Supposedly acting as "quality assurance" consultants, Mazer, Berger, and their accomplices are instead accused of falsifying payments to shell companies, pocketing the proceeds, and making up phony time cards for work they never performed. The defendants have pleaded not guilty.
Bondy, meanwhile, was suspended without pay following the indictments and forced to resign as the head of Bloomberg's Office of Payroll Administration. He could face indictment as well. Bondy, it emerged, not only was a former CityTime consultant, but had also worked with Mazer in the past, yet he didn't disclose those ties until years later.
Originally slated in 1998 to cost $63 million over five years, CityTime has cost the city more than $760 million over its 12 beleaguered years of existence. Despite all that expense, the system is operating in only about a third of all city agencies.
The cost overruns were caused by the vast complexity of the project and changes to the plans, claim Bloomberg officials and the company responsible for building the system, Virginia-based Science Applications International Corp.
Nonsense, says a union official who represents city architects and engineers, and has closely tracked the project.
"There's no way that any problems or changes they had could justify a cost increase of more than 10 times," says Local 375 vice president Jon Forster, who believes SAIC should face criminal investigation. "In 12 years, we haven't changed the number of agencies or the number of employees. My sense is that someone saw a gravy train here, and they said, 'Let's go for it.' "
Last week, federal prosecutors issued subpoenas to SAIC for documents related to the project—presumably to determine just what happened to all that money. There are, records show, a couple hundred other consultants on the project, and you have to wonder what they were doing.
But even after a year of revelations about CityTime, a lot of questions remain unanswered: How much money was wasted or stolen? How did this happen, in spite of red flags dating back to at least early 2003? And how did the city miss it for so long?
Mayor Bloomberg's comments on the scandal have been less than satisfying. Over the years, for one, his administration lowballed the true total cost of the project. And so far he has yet to offer a coherent explanation of how CityTime went bad. It didn't happen by accident, as the mayor appeared to suggest in some of his comments, especially when he called it a "disaster."
The definition of "disaster" generally includes a component of misfortune—as if the project was beset with bad luck. But in this case, there was intent. The indictment makes that clear enough.
In saying he wished he spent more time with the project, Bloomberg appeared out of touch, which is fairly disturbing for a man who cloaks himself as a great manager and technology expert.
He also suggested that the city's trust had been "misplaced." It was almost as if he was characterizing city government as a lover shocked to find out his spouse had cheated on him. The credo "Trust, but verify" comes to mind.
In December, once again seeking to minimize the scandal, he said, "Nothing goes without some problems, whether it's in your family, your company, your government, [or] the world."
Moreover, throughout his administration, Bloomberg repeatedly cut agencies, shrunk city staff, and recited this mantra: "We have to do more with less." But on the CityTime project, he or his aides appeared to have decided it was OK to do less with more.
And for that matter, what does it say about the city comptroller's office, which also had an oversight role on the payroll agency?
In fairness, the current comptroller, John Liu, has aggressively raised questions about CityTime. He was the one who persuaded Mayor Bloomberg to stop the open-ended payments to SAIC, and forced them to agree to a real deadline.
That decision magically accelerated the pace of the project. By last September, just 73,000 employees were on the system. By December, the number had shot up to 100,000 employees, proving that there's nothing like a little work incentive to get things done.
But Liu's success only makes one wonder why questions weren't raised much sooner by his predecessor, William Thompson.
Records indicate that along with the mayor's office, the comptroller's office approved all of the contract increases, but didn't audit CityTime until this year. That audit found, among other things, that the project was plagued by "poor management decisions." It was so bad, in fact, that there was no way to figure out just how many dollars were wasted, the audit said.
Forster, the Local 375 official, tells the Voice that his people approached Thompson's office in 2007 to request an inquiry on CityTime, but were politely rebuffed.
"It just seemed like they had too much on their plate at the time to take on something so massive," Forster says. "And then there was the election and that was sort of it."
Moreover, an SAIC competitor told the New York Post that he had been raising red flags for years, saying he spoke to both Comptroller Thompson's office and the mayor's office about his concerns.
Comptroller spokeswoman Sharon Lee said she couldn't comment on the Thompson era.
And what does the scandal say about the Department of Investigation?
DOI commissioner Rose Gill Hearn has been aggressive in pursuing CityTime this past year. According to the federal indictment, the investigation began in June 2010, when a former consultant told DOI he was being paid by a company that hadn't been approved to work on the project. The federal case appears to rely heavily on the DOI investigation.
But you have to wonder what, if anything, was done before then. It's hard to believe that not one person approached DOI between 1998 and 2010 to complain about the internal workings of the project.
In fact, in 2007, the Voice has learned, DOI did investigate CityTime, including interviewing several CityTime consultants and a few payroll affairs people.
In addition, the probe examined a Department of Defense Inspector General investigation into a number of former SAIC employees who had formally complained that after they blew the whistle on improper billing practices, SAIC retaliated against them by firing them, records show.
What DOI did with the information that was gathered remains unclear.
DOI spokeswoman Diane Struzzi declined to comment on any specifics because of the current, ongoing investigation into CityTime. "There have been a series of allegations going back several years—some have borne fruit and some have not," she said. "There is not a single complaint that came into this department that laid out the kickbacks, the money laundering, or the shell companies. DOI uncovered those crimes."
And, of course, there's the January 2009 anonymous Internet posting, which should have piqued someone's curiosity. It wasn't all that hard to stumble across it.
And then there's the City Council—the supposed check on the mayor's spending habits. While some Council members—like Letitia James and Joseph Addabbo—have sought answers, where was the council in the years when the project grew out of control? Isn't the council supposed to approve big contract increases?
While the council voted to approve the initial contract, James says that it did not get a chance to approve or disapprove the subsequent and sharp cost increases. In essence, she claims, the Bloomberg administration was able to do an end run around on the council's oversight role. Can it really be true that the council had no idea what was happening in the project?
Jamie McShane, a spokesman for Council Speaker Christine Quinn, did not respond to Voice questions.
In many respects, the local media was also behind the curve. Part of the reason was that the Bloomberg administration kept the true cost of the project under wraps. As a result, there was confusion in the press about just what the project cost. A New York Times article in January 2007, for example, set the project cost at $180 million, when, by then, the true expense was already much higher.
It wasn't until Daily News columnist Juan Gonzalez began in late 2009 what would become a series of probing columns that the story began to emerge from the shadows.
Of course, CityTime isn't the first high-tech city project to go horribly awry—just the most expensive and allegedly the most corrupt.
City Comptroller Liu said in May that there were a series of high-tech city projects with cost overruns, missed deadlines, and incomplete or ineffective systems. A city personnel tracking system, for example, mushroomed from $66 million to $155 million.
Just last week, Liu said that a project to create two new 911 emergency systems for the police and fire departments had swelled from $380 million in 2005 to $666 million with no major alterations of the project plans. The project is one year overdue, and has been troubled by poor management and oversight by the contractor, his office said.
He rejected a new $286 million contract request from the Bloomberg administration and demanded a review. The central issue, he charged, was that, similar to CityTime, the project allows a kind of open-ended spending.
"This type of vague budgeting formula allows outside consultants to bill on an hourly basis and collect exorbitant fees, as in the case of the CityTime project," Liu said. "The program was troubled by findings of poor management and less than satisfactory oversight by the original vendor."
The history of CityTime goes back to 1996 and the Giuliani administration. The goal was to computerize the city's payroll system, which covers close to 160,000 workers. After another bidder dropped out, the contract was awarded to MCI Systemhouse. After two years, Systemhouse came up with a thick, heavily detailed, and comprehensive technical plan, or "scope of work," for the project, and estimated that it would cost $63 million and take five years.
In other words, it wasn't like someone just came up with the $63 million budget on a cocktail napkin. A great deal of thought and work went into it.
The proposal contained detailed deadlines that made clear the contractor felt it was possible to complete the job in five years, plus lists of thousands of pieces of high-tech equipment that needed to be purchased and dozens of deadlines listed in 30-day increments.
Even at $63 million, the contract provided fairly lavish pay rates for the contractors: more than $300 an hour for top managers, more than $200 per hour for mid-level people. Even the word processors would get $80 an hour. (A percentage is, of course, kicked to the company.)
Even taking the cost of inflation into account, if the project took 10 years, the finished product still should have cost only about $84.5 million.
Moreover, the contract contained a key provision that levied money damages for missed deadlines—a standard incentive that was irresponsibly eliminated a few years later.
But Systemhouse was bought by WorldCom and dropped the project, transferring its CityTime unit to a new firm, Paradigm 4.
In 2000, the city decided to add new elements to the plan, chiefly switching to a system with one central Internet server, rather than a separate server in each agency. The second of a dozen contract amendments increased the budget to $73 million.
The following year, the city hired Mazer's firm, Spherion, to do something called "quality assurance"—essentially to keep an eye on the project. That contract started at a cost of $3.4 million. Since then, it has been changed a dozen times and has ballooned to $51 million. As the Mazer indictment suggests, not only were they not watching the contractor, no one was watching the watchers.
For some reason, the contract was changed after five months to drop a requirement that Spherion examine and approve each piece of the project. As a result, the last record of such a review dates back to 2001, the comptroller's audit says.
Spherion was also supposed to supply regular summaries of the project's progress. The comptroller could not find any such summaries after 2001.
About a year later, the role of project manager was added to Spherion's contract. That meant that the auditor was now also involved in managing the project, according to the 2010 comptroller's audit—a clear conflict of interest.
By the time Mayor Bloomberg was elected, Paradigm 4 was out of the picture and SAIC had taken over the project. When the contractors changed, critics have said, the council should have had a chance to review it. There was no such review. SAIC was selected without competitive bidding.
And in a critical, and ultimately unfortunate, move, the contract was changed from one that levied penalties for failure to hit milestones to a more open-ended method of monthly payments based on hours worked. Ironically, the move was supposed to "streamline" the process. It caused the opposite effect, giving the contractor no incentive to make deadlines.
SAIC has won many millions in government contracts over the years, but it has also racked up its share of fines. In 1995, the company paid a $2.5 million fine for defrauding the government over a computer system for jet fighters. In 1998, the company paid a $1.1 million fine to the Defense Department.
A group of former employees accused the firm of retaliating against them for complaining about questionable billing practices on another job.
In January 2002, perhaps true to form, SAIC officials had the brilliant idea of trying to hire the city official overseeing them, Richard Valcich. In a letter uncovered from the city comptroller's office, Valcich expressed his dismay at the overture, published reports said.
Valcich said he was sorry "if I seemed rude and abruptly shortened your discussion on a future post city-employment position with SAIC." He added: "It is inappropriate to discuss any post employment with a company that I do business with."
By November 2002, another amendment made more changes to the project, and the approved cost rose again from $115 million.
Meanwhile, things between Valcich and SAIC weren't going so well. In what should have been an enormous red flag to the comptroller and the mayor, Valcich slammed SAIC's job performance in a February 19, 2003, letter released by the comptroller's office.
Valcich accused the company of cutting the city out of the project, creating needless delays, and wasting time and money. He said key personnel weren't showing up at meetings and records weren't being updated. He said the company repeatedly stopped work to "review" progress. He even questioned SAIC's credibility.
"SAIC's commitment to quality is almost nonexistent and is reflected from the top down," Valcich wrote. "This lack of commitment to quality permeates matters both great and small."
He added, "It appears to the city that SAIC seems to think [it] could take as long as possible to finish an activity. . . . The city of New York has spent approximately $35 million on CityTime and does not have a tangible system to show for it."
Despite the almost angry tone of the letter—written by the city's point man on the sprawling project, the man who had been shepherding the project since 1995—it did not appear to have any impact. No one from either the mayor's or comptroller's offices appeared to have jumped in to light a fire under SAIC.
The system kept becoming more complex, and at some point SAIC began to resist going forward. SAIC hired a consultant, Ariel Partners, who warned that the technology the city had paid for couldn't handle all the things the city wanted it to.
SAIC finally said it would go forward, but only if it was paid on an hourly basis, rather than a flat rate per piece of the element. The city agreed, which meant SAIC now had a kind of green light to spend whatever it wanted to make the system happen.
By 2004, Valcich had retired from the payroll post, to be replaced by Joel Bondy, a former consultant with Spherion, the company supposedly vetting the project for the city. Bondy had also worked with Mazer, the recently indicted Spherion consultant, at the city's foster-care agency.
The effort to redesign the system took two more years. An SAIC spokeswoman was now saying the two sides were "working well together, and approved our personnel, approach, and overall architecture."
In 2005, with the redesign complete, the city had a chance to walk away from the project. The city instead turned to Spherion for advice, and, of course, the conflicted firm said, "Let's keep going," the comptroller's audit said.
With a rough squiggle, Bondy signed off on a contract extension that gave SAIC another three to six years to finish the project. The amendment justified the extension only by saying it was needed because of unspecified "delays."
By February 2006, the official contract cost was now at least $225 million, triple the original project cost. This time the increase was blamed on "a major increase in the cost of deployment."
Meanwhile, the alleged fraud involving the Spherion consultants had already begun, the indictment said.
Mazer awarded big-money contracts to friends, who then kicked back $25 million to him. They billed the city for phantom work and squirreled the money away in shell companies. He and his accomplices allegedly pursued this fraud for the next five years, eventually stealing some $80 million.
Meanwhile, the public focus was on the effect of the payroll system on city employees. From 2006 through 2009, unions mounted a campaign to limit the system, especially palm scanners, which were seen as potential health hazards. They also charged that the system violated civil rights, and imposed unfair demands on employees. They held demonstrations, testified at City Council hearings, and spoke to the media. Forster's union filed an unfair labor complaint.
Also in 2007, a former SAIC employee on a federal contract alleged that he and several colleagues were fired for going to the feds to complain about improper billing practices by the company.
A city Vendex form indicated that in June 2007, the Department of Defense Inspector General began investigating SAIC based on those allegations.
Did that disclosure, now more than three years old, trigger any concern about SAIC in city government? It is unclear.
In May 2008, Bondy testified before the City Council. In his written remarks, tellingly, he expended not a word on the cost of the project. Not one word.
But behind the scenes, he repeatedly gave project contractors high performance evaluations in 2008 and 2009, The New York Times reported.
Over the objections of unions and some council members, Bloomberg kept the project going.
In January 2009, the anonymous letter floated onto that CNN bulletin board. In September 2009, Bloomberg added another $140 million to the CityTime budget—bringing the total to more than $700 million.
And then, columnist Juan Gonzalez started writing articles about the project, reporting on December 4, 2009, that Spherion consultant Mitchell Goldstein made $490,000 that year. The top 11 consultants, he wrote, made an average of $307,000 a year through 2012.
On December 17, he reported that Bondy convinced Spherion to hire Goldstein—a possible conflict of interest because the two men had been business partners.
Gonzalez quoted a former consultant who said he hardly did any real work, but got paid $120,000 for it. "The unwritten rule was to keep billing for the hours you showed up, not the work you did," this consultant told Gonzalez.
And even after the disclosure that Bondy had close ties to the people he was supposed to be overseeing, Bloomberg left him in office for another year.
In January of last year, Gonzalez reported that a tech consultant who used to work for the city got millions from CityTime while running a full-time lobbying firm.
Public pressure began to take hold, and the story started to move forward rapidly.
In February, Liu rejected an $8 million Spherion contract, and launched an audit of CityTime. The budget was now $722 million. He called on the mayor to freeze all contracts, calling it a "money pit."
In March, Bloomberg made his admission that the project was a "disaster," but insisted such projects were rarely successful. A few days later, he said he wished he had "focused on it more."
On March 26, Gonzalez reported that the CityTime project manager made $650,000 in 2009, and there were 39 other people making at least $500,000 a year. He noted that the city laid off 510 public school aides to save $12 million, while earmarking $24 million to pay CityTime consultants.
"How can anyone justify firing $18,000-a-year school aides while hiring half-a-million-dollar computer geeks who can't even deliver a good product?" Gonzalez wondered.
But Bloomberg's people continued to defend the need for the project and granted it an extension through September.
In June, the DOI investigation began, with that complaint from the consultant paid by a company not approved to work on the project.
By last August, Bloomberg and Liu were sparring over the contract. Liu didn't want to approve another $176 million in spending; Bloomberg was threatening to withhold paychecks.
The city Independent Budget Office reported then that $641 million in capital funds were either spent or planned, along with $232 million in spent and planned operating funds. That added up to more than $873 million.
Last September, Liu refused to pay $32 million to SAIC until next June unless the company finished the project by then. Finally, there was a penalty attached to missing a deadline: $3 million per month.
The indictments came down on December 15. The six defendants were released on bail. Authorities seized some $850,000 from safety deposit boxes.
Bondy was suspended the following day and resigned three days before Christmas. The Spherion contract was suspended. The Department of Education suspended a separate Spherion contract. Bloomberg ordered a review of technology projects.
The state comptroller jumped on the bandwagon, rejecting an SAIC contract. And now, with the federal and city investigations still ongoing, more indictments could be coming. But taxpayers have already paid for everyone else's mistakes, inattention, and misbehavior.
Get the ICYMI: Today's Top Stories Newsletter
Catch up on the day's news and stay informed with our daily digest of the most popular news, music, food and arts stories in New York, delivered to your inbox.