S ince 2001, America’s intelligence agencies have been doubly damned. First they were deemed incompetent for their failures on September 11 and for the infamous 2002 National Intelligence Estimate (NIE) on the Iraqi weapons of mass destruction that weren’t, then they were tarnished by the epithets of scandal labeled "Guantánamo," "torture," and "spying on Americans." They thus face the challenge of reshaping themselves and restoring their social contract with the American people–in which the American people understand that they cannot know all the details of what intelligence agencies do in their name but require that those agencies respect the limits of American values–before they can become fully functioning tools for national security.

These failures have powered thousands of hours of hearings and thousands of pages of planning. But so far, the carousel of reorganization has produced more shuffle than substance. To truly reshape intelligence gathering in order to meet today’s threats, all of the Cold War legacy must be put on the table: organizations, security practices, and, especially, connections to U.S. society. As the intelligence community reshapes itself, and particularly as it needs to collect more information at home against a changed threat, it will confront a paradox: Domestic intelligence will be acceptable only with more transparency, but transparency can tip off would-be targets about how to stay below the radar. What is required are mechanisms for accountability and oversight to serve as surrogates for what the American people would find acceptable if they could only know.

Driving the reshaping of intelligence is the changing nature of the threat. The change can hardly be overstated. To be sure, some states, like North Korea or China or Russia, are still important targets for intelligence. But transnational targets, like terrorists or criminals, which were secondary before are now primary. The Soviet Union was a relatively bounded problem. It was glacial. It was not likely to surprise. The only surprise came when its glaciality became too onerous, and it collapsed suddenly. Cold War intelligence targeted big things with addresses, like Soviet armies, that weren’t likely to change fast. Now, the priority is small things without addresses, which may change unpredictably as new groups or attack modes arise. September 11 drove home just how much damage a small group of determined men could do.

Three other differences are more consequential still. We had some "story" about states, even states as different from us as the Soviet Union. They were geographic, bureaucratic, and hierarchical. As a result, intelligence had a framework in which to locate new information, and communicating with policymakers was relatively easy because they shared the same frame. Even more important, in the old world of intelligence, the problem could be taken to be understanding "over there." While we hoped to influence Soviet behavior, we didn’t expect that we would. So the challenge was understanding them on their terms. As former Secretary of Defense Harold Brown quipped about the U.S.-Soviet nuclear competition, "When we build, they build. When we cut, they build."

But terrorists are utterly different. They are constantly adapting to us and our vulnerabilities. The September 11 terrorists weren’t airplane buffs; rather, they had done enough good tactical reconnaissance to know their plan would succeed. They had found the seams in our system. Today, it’s not enough to know about them; intelligence can’t understand them without knowing a lot abut "us"–an especially uncomfortable fact for those agencies, like the CIA and NSA, which have long been enjoined from working on or in the United States.

Moreover, many intelligence questions about terrorists are "complexities," not puzzles or mysteries. Puzzles have clear solutions. Mysteries are unknowable, but they have some shape; we know what variables matter most in producing an outcome, and we may have some historical evidence about how they interact. The Soviet Union was a puzzle: There was a lot we didn’t know, but we had a general idea of how things should look, and so as we found new puzzle pieces, we knew where to fit them. Complexities, though, are mysteries-plus. Large numbers of relatively small actors respond to a shifting set of situational factors–groups forming and reforming, seeking to find vulnerabilities, thus adapting constantly, and interacting in new ways. There may be no historical patterns in either what to look for or how the critical factors interact. Dealing with complexities in the new world of intelligence will require a dramatic rethinking.

The two intelligence failures of the early 2000s led to the creation of the Department of Homeland Security (DHS), which was mostly operational in intent, trying to produce some coherence among the various agencies responsible for protecting America’s borders and responding to internal disasters. For intelligence, there were three primary changes. First was the reorientation of the FBI’s mission. Second, the Intelligence Reform and Terrorism Prevention Act of 2004 set up the National Counterterrorism Center (NCTC), suggesting a new model for how intelligence did its work. Finally, that act also created the director of national intelligence (DNI) in an effort to build "jointness" across the 16 agencies of the intelligence community. All of these were steps forward, but all of them failed to change the nature of our intelligence structure, and so ultimately failed in improving our intelligence capabilities.

The 9/11 attacks prompted immediate calls to establish a new domestic intelligence service, separate from the FBI. The 9/11 Commission’s diagnosis pointed straight at the limitations imposed by the FBI’s culture of case-based law enforcement, saying that FBI agents were "trained to build cases, [and] developed information in support of their own cases, not as part of a broader more strategic [intelligence] effort." Quite literally, while the Bureau did traditional counterintelligence against the spies of foreign nations, in its primary law enforcement mission, if information wasn’t relevant to a criminal case at hand, it wasn’t considered worth saving. As a result, captured terrorist handbooks went untranslated. Likewise, sharing information, even with intelligence agencies, risked compromising existing cases, and so rarely occurred.

FBI director Robert Mueller, who had been in the post for one week on September 11, moved quickly to reorient the Bureau toward prevention and intelligence; his reorganization plan reached Congress in November 2001. For counterterrorism purposes, he centralized the Bureau’s famously autonomous field offices. As Arthur Cummings, now head of the National Security Branch, explained, for today’s Bureau, "There is no such thing as a local terrorism problem. Something might happen locally, but within two seconds, you discover national and international connections." The Bureau’s website advertises its top priority as "protect[ing] the United States from terrorist attack." The FBI’s budget more than doubled between 2001 and 2008, from $3.1 to $6.4 billion. It increased from 34 to 101 the number of joint terrorism task forces (JTTFs), which bring together FBI agents, state and local law enforcement officials, and representatives from other federal agencies to investigate terrorism cases.

This transformation is indeed a sea-change. When I oversaw NIEs in the 1990s, I knew the FBI through intelligence, but that was like trying to understand the National Football League by talking to the placekickers: Intelligence was important to the Bureau, but not central. It was labeled "support" (if not "furniture"), and the label was apt. Intelligence then meant mostly operational support: can you find this suspect’s address? Now, the Bureau is trying to make intelligence-led prevention its mission across the organization. Those cases that earlier constrained what information was collected are meant to become, instead, platforms for collecting intelligence. It is an impressive goal and a revolution in organizational culture.

But does it work? In 2005, as part of its reconfiguration, the Bureau put its beefed-up Office of Intelligence back together with Counterintelligence and Counterterrorism in the National Security Branch (NSB), thus creating an intelligence branch with a similar relationship to the DNI as that of the NSA–i.e., not managed day-to-day by the DNI, but looking to that office for a budget and broad guidance. In effect, the FBI created a service-within-a-service.

On the plus side, this shows the FBI really does value intelligence work, a big change from before. The bad news is that while the counterterrorism squads think the work is important, they often do not like it. It is relentless, chasing one lead after another, without the closure of arrests. It’s not what many in the Bureau signed up for. In short, the Bureau has changed, but that change is still very much a work in progress.

For now, Congress and the body politic have decided that Mueller and his colleagues deserve time to see if they can reshape the Bureau. Another major terrorist attack on the United States, however, would raise the question of whether the Bureau really could transform or if, instead, the nation required a new domestic intelligence agency.

The National Counterterrorism Center