The Transportation Security Administration (TSA) released Friday a previously confidential report by the Secure Flight Working Group (SFWG), an independent expert committee on which I served. The committee’s charter was to study the privacy implications of the Secure Flight program. The final report is critical of TSA’s management of Secure Flight.
(Besides me, the committee members were Martin Abrams, Linda Ackerman, James Dempsey, Daniel Gallington, Lauren Gelman, Steven Lilienthal, Bruce Schneier, and Anna Slomovic. Members received security clearances and had access to non-public information; but everything I write here is based on public information. I should note that although the report was meant to reflect the consensus of the committee members, readers should not assume that every individual member agrees with everything said in the report.)
Secure Flight is a successor to existing programs that do three jobs. First, they vet air passengers against a no-fly list, which contains the names of people who are believed to pose a danger to aviation and so are not allowed to fly. Second, they vet passengers against a watch list, which contains the names of people who are believed to pose a more modest danger and so are subject to a secondary search at the security checkpoint. Third, they vet passengers’ reservations against the CAPPS I criteria, and subject those who meet the criteria to a secondary search. (The precise CAPPS I criteria are not public, but it is widely believed that the criteria include whether the passenger paid cash for the ticket, whether the ticket is one-way, and other factors.)
The key section of the report is on pages 5-6. Here’s the beginning of that section:
The SFWG found that TSA has failed to answer certain key questions about Secure Flight: First and foremost, TSA has not articulated what the specific goals of Secure Flight are. Based on the limited test results presented to us, we cannot assess whether even the general goal of evaluating passengers for the risk they represent to aviation security is a realistic or feasible one or how TSA proposes to achieve it. We do not know how much or what kind of personal information the system will collect or how data from various sources will flow through the system.
The lack of clear goals for the program is a serious problem (p. 5):
The TSA is under a Congressional mandate to match domestic airline passenger lists against the consolidated terrorist watch list. TSA has failed to specify with consistency whether watch list matching is the only goal of Secure Flight at this state. The Secure Flight Capabilities and Testing Overview, dated February 9, 2005 (a non-public document given to the SFWG), states in the Appendix that the program is not looking for unknown terrorists and has no intention of doing so. On June 29, 2005, Justin Oberman (Assistant Administrator, Secure Flight/Registered Traveler [at TSA]) testified to a Congressional committee that “Another goal proposed for Secure Flight is its use to establish “Mechanisms for … violent criminal data vetting.” Finally, TSA has never been forthcoming about whether it has an additional, implicit goal – the tracking of terrorism suspects (whose presence on the terrorist watch list does not necessarily signify intention to commit violence on a flight).
The report also notes that TSA had not answered questions about what the system’s architecture would be, whether Secure Flight would be linked to other TSA systems, whether and how the system would use commercial data sources, and how oversight would work. TSA had not provided enough information to evaluate the security of Secure Flight’s computer systems and databases.
The report ends with these recommendations:
Congress should prohibit live testing of Secure Flight until it receives the following from the [Homeland Security Secretary].
First, a written statement of the goals of Secure Flight signed by the Secretary of DHS that only can be changed on the Secretary’s order. Accompanying documentation should include: (1) a description of the technology, policy and processes in place to ensure that the system is only used to achieve the stated goals; (2) a schematic that describes exactly what data is collected, from what entities, and how it flows though the system; (3) rules that describe who has access to the data and under what circumstances; and (4) specific procedures for destruction of the data. There should also be an assurance that someone has been appointed with sufficient independence and power to ensure that the system development and subsequent use follow the documented procedures.
In conclusion, we believe live testing of Secure Flight should not commence until there has been adequate time to review, comment, and conduct a public debate on the additional documentation outlined above.
Speaking for myself, I joined the committee with an open mind. A system along the general lines of Secure Flight might make sense, and might properly balance security with privacy. I wanted to see whether Secure Flight could be justified. I wanted to hear someone make the case for Secure Flight. TSA had said that it was gathering evidence and doing analysis to do so.
In the end, TSA never did make a case for Secure Flight. I still have the same questions I had at the beginning. But now I have less confidence that TSA can successfully run a program like Secure Flight.
I’d like to think that members of the SFWG are somewhat more accessible to the public than the TSA. Members may not be free to discuss the details of the committee, but at least we know who they are and we can bring our concerns to their attention. They are not bound to act on our concerns, but at least they may act as a voice for the public. I wouldn’t know how to communicate my concerns to the right bureaucrat at the TSA.
I do think, though, that all of this secrecy does not serve the public well. A business owner would probably be foolish to invest in a proprietary cryptographic technology where the provider refused to state anything about the protocol. As a taxpayer, I feel a bit foolish allowing my government to invest in “technology” that is supposed to protect me, when I don’t know anything about what I’m getting for my money. I suppose I could say the same thing about nuclear weapons, but then I don’t expect nuclear weapons to be used on a day-to-day basis to make decisions about my life.
I’m quite sure that if I could tell you everything, you would agree that although Secure Flight may be a reasonable idea in the abstract, Secure Flight as run by TSA is a different story.
No doubt you wrote this without intending even the slightest hint of irony. But if your goal was to “break the symmetry”….
Still, I’m inclined to sympathize with government officials who find themselves forced to say things like this, so I’m not going to press you any harder than I’d press them. And I’m prepared to believe that your working group found real problems with Secure Flight, apart from any overblown privacy concerns it may–or may not–have harped on.
But perhaps I’ve helped you understand a bit better how difficult it can be for government security people to have to defend their work against hostile, suspicious, agenda-driven critics who invariably choose the least charitable possible interpretation of whatever they’re allowed to know–and then assume the worst about anything they’re not allowed to know.
Dan,
Let me break the symmetry for you.
The committee members were chosen by TSA. There is a formal TSA document establishing the committee and giving it a specific charter; but TSA says that document is confidential. The committee’s meetings and deliberation are secret because TSA says so. (They tell us that breaches might lead to criminal penalties.)
The committee’s job was not political advocacy. There are many empirical questions about Secure Flight. What is the false positive rate? What is the false negative rate? What measures have been taken to improve those rates? Are those measures effective? What use will be made of commercial data? How would the use of commercial data affect the false positive and false negative rates? How much does all of this cost? What measures have been taken to protect the Secure Flight systems and databases from break-ins?
I thought the committee would be considering those questions. One would expect a reasonably-run program to have collected data on some of these questions. Yet TSA had almost no data or analysis to offer on these issues. Hence the committee’s frustration.
Concerning the effect of using commercial data, TSA did large-scale tests in the winter and spring of 2005. A TSA official made assertions about the test results in Congressional testimony in June 2005. Yet as of late August 2005, TSA claimed it had no test results to show the committee. (Evaluating the results of these tests had been one of committee’s TSA-defined goals.)
I wish I could say more, but I am hamstrung by TSA’s confidentiality rules. I’m quite sure that if I could tell you everything, you would agree that although Secure Flight may be a reasonable idea in the abstract, Secure Flight as run by TSA is a different story.
Interesting….the working group found that both the goals and the methods of Secure Flight are ill-defined–or perhaps just largely undisclosed; that it was established, and appears intended to operate, with little oversight; and that its potential links with other government security initiatives with ostensibly unrelated goals were worrisome. The group also produced a list of questions which it believes should be, but have not yet been answered, and whose answers it implicitly worries will not be to its liking.
By the same token, I find that the working group’s goals and methods are ill-defined–or perhaps just largely undisclosed; that it was established, and apparently operated, with little oversight; and that its potential links with other politicized privacy initiatives with ostensibly unrelated goals are worrisome. I could offer my own list of questions for the working group that I believe should be, but have not yet been, answered, and I rather suspect I wouldn’t like the answers, even if the working group were willing to give them to me.
Then again, I suppose the parallels shouldn’t be surprising. Privacy advocacy is a lot like homeland security, after all–both are battling a widespread, secretive, little-understood foe with infinitely evil intentions, constantly changing form and methodology, and a stunningly broad scope that might well include the most innocuous-seeming people and institutions. In such a battle, it’s important to suspect everyone and everything, to avoid tipping one’s hand to the enemy regarding methods or even intentions–and, most important of all, never, ever to compromise even one inch for the sake of expediency, realism or some superseding goal.
The only difference I can see is that the Department of Homeland Security’s enemy is, to the best of my knowledge, not a chimera.
Hello Professor Felton, thanks for your opinions on the SFWG release, your colleague Bruce Schneier also posted his thoughts on Secure Flight, much of the same critiques.
Couple of typos there: realisstic, Capabilityes (I hope), descrubtion.
[Fixed. Thanks. — EF]