They thought they were making routine purchasesthe innocent, everyday pickups of charcoal and hummus, bleach and sandwich bags, that keep the modern household running. Regulars at a national grocery chain, these thousands and thousands of shoppers used the store's preferred-customer cards, in the process putting years of their lives on file. Perhaps they expected their records would be used by marketers trying to better target consumers. Instead, says the company's privacy consultant, the data was used by government agents hunting for potential terrorists.
The saga began with a misguided fit of patriotism mere weeks after the World Trade Center and Pentagon attacks, when a corporate employee handed over the recordsalmost literally, the grocery liststo federal investigators from three agencies that had never even requested them. In a flash, the most quotidian of exchanges became fodder for the Patriot Act.
When the company's legal counsel discovered the breach, she turned for advice to Larry Ponemon, CEO of the consulting firm Privacy Council and a former business ethics professor at Babson College and SUNY. "I told her it's better to be transparent," Ponemon recalls. "Send a notice to loyalty cardholders telling them what happened. She agreed and presented that to the board but they said, 'No, we don't want to hand a smoking gun to litigators.' " The attorney, who has since resigned from the grocery chain, declined through Ponemon to be interviewed or to identify herself or her former employer. To this day, the customers haven't been informed.
"It wasn't a case of law enforcement being egregiously intrusive or an evil agency planting a bug or wiretap. It was a marketing person saying, 'Maybe this will help you catch a bad guy,' " Ponemon says.
As John Ashcroft's Citizens Corps spy program prepares for its debut next month, it seems scores of American companies have already become willing snitches. A few months ago, the Privacy Council surveyed executives from 22 companies in the travel industrynot just airlines but hotels, car rental services, and travel agenciesand found that 64 percent of respondents had turned over information to investigators and 59 percent had lowered their resistance to such demands. In that sampling, conducted with The Boston Globe, half of the businesses said they hadn't decided if they'd inform customers of the change, and more than a third said outright that they wouldn't. Only three said they would go public about the level of their cooperation with law enforcement.
The final destination of all that data scares Ponemon and other civil libertarians, defenders of the Fourth Amendment prohibition on unreasonable search and seizure. Ponemon, for one, suggests federal authorities are plugging the information into algorithms, using the complex formulas to create a picture of general-population trends that can be contrasted with the lifestyles of known terrorists. If your habits match, expect further scrutiny at the least.
"I can't reveal my source, but a federal agency involved in espionage actually did a rating system of almost every citizen in this country," Ponemon claims. "It was based on all sorts of informationpublic sources, private sources. If people are not opted in"meaning they haven't chosen to participate"one can generally assume that information was gathered through an illegal system."
After crunching those numbers through the algorithm, he says, its creators fed in the files of the 9-11 terrorists as a test. "The model showed 89.7 percent accuracy 'predicting' these people from rest of population," Ponemon reports.
Oddly enough, "one of the factors was if you were a person who frequently ordered pizza and paid with a credit card," Ponemon says, describing the buying habits of a nation of college students. "Sometimes data leads to an empirical inference when you add it to other variables. Whether this one is relevant or completely spurious remains to be seen, but those kinds of weird things happen with data."
The thirst for consumer records is bipartisan. In April, Bill Clinton told the BBC that when it comes to fighting terrorism, "more than 95 percent of the people that are in the United States at any given time are in the computers of companies that mail junk mail, and you can look for patterns there."
Katherine Albrecht, a crusader against grocery loyalty cards and invasive marketing, notes in a paper to be published in the Denver Law Review, "Virginia Congressmen Jim Moran (D-VA) and Tom Davis (R-VA) recently introduced legislation that would require all states' driver's licenses and ID cards to contain an embedded computer chip capable of accepting 'data or software written to the license or card by non-governmental devices.' " The mandatory "smart chips" would carry bank and debit card data so that citizens could use their ID cards "for a variety of commercial applications." Even library records, shopping coupons, and health records could be stored on the chips.
Adding to this vision of technological dystopia, companies are already developing cameras and other scanners that can seamlessly trace individuals as they wander through stores, going so far as to zoom in on their faces should they linger over an item, to provide retailers with ever more data.
The problem is that, as with the link between take-out pizza and terrorism, statistics don't always prove cause and effect. Mathematician Karen Kafadar of the University of Colorado at Denver explains that such a finding is "a proxy. It just happened to have something that correlated. There's actually something else going on but it's an indicator, like drinking beer and lung cancer might be. Beer doesn't cause lung cancer, but people drinking a lot of beer might also be smoking."
Ponemon is more concerned about process than the data itself. "Total privacy does shelter bad guys, there's no question about that. But transparency is also good," he argues. "There should be some labeling or notice." In theory, consumers and investors could punish offending companies by channeling their money elsewhere. Without honest managers, though, the free market's self-correcting mechanism never gets a chance to kick in.
Librarians have filled their listservs with e-mails sharing strategies for resisting law enforcement attempts to grab hold of their users' book lists. But the corporate world doesn't foster that kind of purist culture. When the Federal Bureau of Investigation came knocking for the names of scuba divers this spring, the Professional Association of Diving Instructors forked over a roll of more than 2 million certified divers without so much as being served a subpoena.
The feds were acting on no specific threat, just a hunch that someone might attack that way. And again, these data dumps are just attempts to do good. Would Attorney General John Ashcroft's new TIPS campaignthe Terrorism Information and Prevention Systemencourage people like the mole at the grocery store chain to spill info into the tanks of unethical investigators?
The Department of Justice, which seeks informants in utility, cable, and other such industries operating in communities, denies that it will cultivate sources placed in data-mining operations. "This makes TIPS sound so much more sophisticated than it's going to be," says spokesperson Charles Miller. "This is still in development but it's nothing more than something to make people more aware of what's going on around them, and most people do that now anyway."
Likewise, both the Federal Bureau of Investigations and the Central Intelligence Agency denied roles in any sweeping algorithm to measure citizens' potential terrorist leanings. If anything, the FBI has recently been taken to task for being a tin-cans-and-string Luddite organization. But the FBI is a client of the consumer data aggregator ChoicePoint. And a U.S. official tells the Voice, "Can I categorically deny anybody in government is doing it? No."
An admission that the government is combing through purchase records certainly would help explain why, according to the Naples Daily News, federal agents reviewed the shopper-card transactions of hijacker Mohammed Atta's crew to create a profile of ethnic tastes and terrorist supermarket-shopping preferences.
Algorithms are already used to search for things as diverse as credit card fraud and ideal college applicants. Since 1998, airline ticket buyers have been sifted at the reservations desk by the Computer Assisted Passenger Prescreening System, or CAPPS, a net championed by Al Gore and set to expand dramatically. The group overseeing the algorithm, the Transportation Security Administration, won't comment on what new data might be added to create CAPPS 2.
"At a conceptual level, the work that these algorithms do is not much different than the work that a detective undertakes in assessing whether an individual is a suspect in a crime," explains Christy Joiner-Congleton, CEO of Stone Analytics, a leading developer of such programs. "Good algorithms sort through mountains of outcomes and possible contributing factors and identify relationships for very rare events, like terrorism. The more exotic the outcome, the more data is needed to discover it, and the more sophisticated the algorithm must be to discover it."
Academic mathematicians and statisticians who design algorithms have also called for broader databases. Among them are Kafadar and Max D. Morris of Iowa State University, co-authors of a new paper titled "Data-Based Detection of Potential Terrorist Attacks on Airplanes." They note that "[a]fter the fact, some common elements of the suspected terrorists are obvious: None were U.S. citizens, all had lived in the U.S. for some period of time, all had connections to a particular foreign country, all had purchased one-way tickets at the gate with cash. The statistical odds that five out of 80 revenue passengers (in the case of one of the four hijacked flights on September 11) fit this profile might, by itself, be unusual enough to warrant concern."
Racial profiling finds quasi-acceptance in the hunt for terrorists, as it does in the drug war or the pursuit of serial killers, who tend to be middle-aged white men. But Kafadar and Morris argue that the "historical data must be relevant to a specific flight. For example, a United flight leaving San Francisco for Seoul, Korea, could be expected to carry a much larger fraction of Asian passengers than one might see on a flight from, say, Des Moines to Denver," the authors write. A trip like Atta's, Kafadar tells the Voice, "wasn't a flight coming from Saudi Arabia. There were a disproportionately high number of Arabic names given about 80 people to choose from."
But the algorithm method didn't fail on 9-11the human response did. When the screening program spotted something unusual about at least one of the flights, the people in charge elected only to reinspect the luggage. According to The Wall Street Journal, CAPPS tagged hijackers Nawaf Alhazmi and Khalid Al-Midhar because they'd reserved their tickets by credit card, but paid in cash. The right-wing National Review slammed CAPPS for failing to include race, religion, and national origin in its calculations or to tie the system into manual searches of passengers, and not just baggage.
MIT mathematician David R. Karger says harassing individuals is foolhardy, but so is refusing to consider sensitive demographics. "This is just making your predictive capability worse," he writes in an e-mail interview. "Much more appropriate is to use the best data you've got, but to remember that probability doesn't mean certainty."
Joiner-Congleton writes, "Fundamentally, the algorithms themselves (if created in a technically correct fashion) are not the thing to fear. Rather, as in life, the things to fear are the conclusions drawn and the subsequent actions taken. Nevertheless, drawing conclusions from data is a necessary thing in life. People must do this to survive. Imagine the havoc that would be wreaked on the roads of America if we ignored the sounding of a horn on the freeway. Horn-blowing is usually associated with a dangerous event. We ignore it at our peril."
She even conceives of developing algorithms so advanced that society might intervene, to get people liable to be recruited into cells back on track before they can be seduced by elements like Al Qaeda. "There is a possibility that with sufficient information about known terrorists we could evolve to the point where we could spot terrorists in the making," she argues. "We believe that individuals can be at risk of becoming drug addicts, or joining gangs, or having affairs, or any number of things at certain times and under certain conditions in their lives. . . . Thorough and continued algorithmic investigation of terrorist behavior is very likely to shed light on their origins, and possibly lead to proactive efforts."
But there's a truly slippery slope here. We live in a nation that for months has held at least 700 peopleand possibly hundreds moreincommunicado, with no more solid connection to terrorism than that they were born in Middle Eastern countries.
Privacy may seem like a luxury in a nation at war, but that moral concept lies at the heart of constitutionally guaranteed liberties. That's why so many people are willing to fight for it. A lawsuit filed by John Gilmore, an early employee of Sun Microsystems, aims to restore the anonymity central to the freedom to travel in America. He names Ashcroft, FBI director Robert Mueller, and security czar Tom Ridge as defendants, among other officials, along with two airlines. Gilmore wants to prevent security at airports from demanding identification from him, or subjecting him to arduous and invasive searches when he refuses to provide a photo ID. The emphasis, he says, should be on strengthening cockpits and developing "fly by wire" systems to automatically land planes under threat. But our terrorism fears extend well past airlines to water-tainting, dirty bombs, suicide bombers, conventional bombing, or even simply opening fire with an assault weapon in Grand Central Stationthe kinds of attacks that are difficult to prevent in an open society.
For now, we rely on tools like algorithms, and algorithms make mistakes. Albrecht notes that in a three-month test period, the Department of Defense investigated 345 employees after a program falsely fingered them for abusing shopping privileges. In another case, an elderly woman was repeatedly stopped and questioned in airports because her name matched that of a young man already in prison for murdera glitch that may indicate CAPPS or another algorithm is using data illegally, for basic criminal investigation and not anti-terrorism. Further, supermarket records have been seized by Drug Enforcement Agency investigators looking for purchases of small plastic baggies, often used in the drug trade, Albrecht observes.
"I am not a number!" shouted Patrick McGoohan, star of the British TV show The Prisoner, when he rejected life in an idyllic village where he was held and constantly monitored. "I am a free man." Now that this nation is at war with terror, perhaps you'll remain free as long as your "Potential Terrorist Quotient" remains low enough.
Get the ICYMI: Today's Top Stories Newsletter
Catch up on the day's news and stay informed with our daily digest of the most popular news, music, food and arts stories in New York, delivered to your inbox.