The New York Times
A survey of federal agencies has found more than 120 programs that collect and analyze large amounts of personal data on individuals to predict their behavior.
The survey, to be issued Thursday by the , an investigative arm of Congress, found that the practice, known as data mining, was ubiquitous.
In canvassing federal agencies, the accounting office found that 52 were systematically sifting through computer databases. These agencies reported 199 data-mining projects, of which 68 were planned and 131 were in operation. At least 122 of the 199 projects used identifying information like names, e-mail addresses, Social Security numbers and driver's license numbers.
The survey provides the first authoritative estimate of the extent of . It excludes most classified projects, so the actual numbers are likely to be much higher.
The Defense Department made greatest use of the technique, with 47 data-mining projects to track everything from the academic performance of Navy midshipmen to the whereabouts of ship parts and suspected terrorists.
Senator Daniel K. Akaka, Democrat of Hawaii, who requested the report by the accounting office, said: "I am disturbed by the high number of data-mining activities in the federal government involving personal information. The government collects and uses Americans' personal information and shares it with other agencies to an astonishing degree, raising serious privacy concerns."
A federal advisory committee appointed by Defense Secretary Donald H. Rumsfeld said last week that Congress should pass laws to protect the civil liberties of Americans when the government and data files for information about terrorists.
Newton N. Minow, chairman of the committee, said he and other panel members would formally present their report to Rumsfeld on Thursday.
The panel said federal agencies should generally be required to obtain court approval "before engaging in data mining with personally identifiable information" on United States citizens. It also said that federal investigators should, if possible, work with anonymous data, stripped of personal identifiers, and use the minimum amount of data needed to achieve their purpose.
The panel was created to quell a over a Pentagon plan to hunt terrorists by monitoring e-mail messages and fishing through huge databases of financial, medical and travel information.
The accounting office defined data mining as the use of sophisticated technology, statistical analysis and modeling to uncover hidden patterns and subtle relationships in data, and to infer rules that allow for the prediction of future activity.
Of the 199 data-mining projects, 54 use information from the private sector, like credit reports and records of credit card transactions. Seventy-seven projects use data obtained from other federal agencies, like student loan records, bank account numbers and taxpayer identification numbers.
In its catalog of data mining, the accounting office listed these projects:
The Internal Revenue Service mines financial data to predict which individual tax returns have the greatest potential for fraud and which corporations are most likely to make improper use of tax shelters.
The Defense Intelligence Agency mines data from the intelligence community and searches the Internet to identify people, including United States citizens, who are most likely to have connections to foreign terrorist activities.
The Department of Homeland Security seeks clues to possible terrorist activity by looking for patterns in myriad records of crimes, arrests and unusual behavior, traffic tickets and incidents involving the possession of firearms.
James X. Dempsey, executive director of the , a civil liberties group, said: "In many cases, the private sector is subject to stricter standards than the government. The Fair Credit Reporting Act, for example, imposes limits on commercial uses of personal financial and other data, but there are virtually no limits on government uses."
Entire contents, Copyright © 2004 The New York Times. All rights reserved.