In August, an entity calling itself the “Shadow Brokers” took the security world by surprise by publishing what appears to be a portion of the NSA’s hacking toolset. Government investigators now believe that the Shadow Brokers stole the cache of powerful NSA network exploitation tools from a computer located outside of the NSA’s network where they had been left accidentally, according to Reuters. A new detail, published for the first time in yesterday’s Reuters report, is that the NSA learned about the accidental exposure at or near the time it happened. The exploits, which showed up on the Shadow Brokers’ site last month, target widely used networking products produced by Cisco and Fortinet and rely on significant, previously unknown vulnerabilities or “zero days” in these products. The government has not officially confirmed that the files originated with the NSA, but the Intercept used documents provided by Edward Snowden to demonstrate links between the NSA and the Equation Group, which produced the exploits.

The Reuters story provides a partial answer to the most important question about the Shadow Brokers leak: why did the NSA seemingly withhold its knowledge of the Cisco and Fortinet zero days, among others, from the vendors? According to unnamed government sources investigating the matter, an NSA employee or contractor mistakenly left the exploits on a remote computer about three years ago, and the NSA learned about that mistake soon after. Because the agency was aware that the exploits had been exposed and were therefore vulnerable to theft by outsiders, it “tuned its sensors to detect use of any of the tools by other parties, especially foreign adversaries with strong cyber espionage operations, such as China and Russia.” Apparently finding no such evidence, the NSA sat on the underlying vulnerabilities until the Shadow Brokers posted them publicly.

But the NSA’s overconfidence should disturb us, as security researcher Nicholas Weaver points out. The “sensors” mentioned by Reuters are likely a non-technical reference to monitoring of the Internet backbone by the NSA under such authorities as Section 702 and Executive Order 12333, which could act as a form of Network Intrusion Detection System (NIDS). (The Department of Homeland Security also operates an NIDS called Einstein specifically to monitor government networks.) But Weaver explains that at least some of the exploits, including those that affected Cisco and Fortinet products, appear not to lend themselves to detection by outside monitoring since they operate within a target’s internal network. In other words, the NSA’s confidence that its surveillance tools weren’t being used by other actors might have been seriously misplaced.

The NSA’s decision not to disclose the Cisco and Fortinet vulnerabilities becomes even more questionable in light of the fact that some of the specific products affected had been approved by the Department of Defense’s Unified Capabilities (UC) Approved Products List (APL), which identifies equipment that can be used in DoD networks:1

Under National Security Directive 42 [.pdf], NSA is tasked with securing “National Security Systems” against compromise or exploitation, a mission which was traditionally housed within the Information Assurance Directorate (IAD). The NSA is currently in the process of combining the “defensive” IAD with its “offensive” intelligence-gathering divisions, but high-level officials charged with information assurance have acknowledged the NSA’s defensive mission is more important than ever. Regardless of whether the mission of protecting National Security Systems is interpreted broadly or narrowly, the NSA’s failure to remedy defects in products used widely across the IT sector and apparently by the government, and even the DoD itself, is difficult to defend.

Above all, the Shadow Brokers story highlights the need for oversight of the government’s use of zero days. Right now, the decision whether to retain or disclose a vulnerability is theoretically governed by the Vulnerabilities Equities Process (VEP), a once-secret policy that EFF obtained in redacted form via a Freedom of Information Act lawsuit. But because the VEP isn’t binding on the government, as far as we can tell, it’s toothless. While we don’t know the exact considerations employed by the government in reaching a decision to withhold a zero day, several of the high-level considerations described by White House Cybersecurity Coordinator Michael Daniel in a blog post about the VEP seem highly relevant:

• How much is the vulnerable system used in the core Internet infrastructure, in other critical infrastructure systems, in the U.S. economy, and/or in national security systems?
• Does the vulnerability, if left unpatched, impose significant risk?
• How much harm could an adversary nation or criminal group do with knowledge of this vulnerability?
• How likely is it that we would know if someone else was exploiting it?

Even if NSA initially believed the specific vulnerabilities at issue in this case wouldn’t be discovered by others, its knowledge that the exploits had been left exposed should have changed that calculus. And if NSA knew specifically that the exploits had been stolen, it’s hard to think of a rationale where disclosure would still be outweighed by other considerations. Coincidentally, the NSA seems to have lost control of the Shadow Brokers exploits in 2013, during a fallow period for the VEP. Although the VEP was written in 2010, Michael Daniel told Wired that it was not “implemented to the full degree that it should have been” and was only “reinvigorated” in 2014.

We think lawmakers should be concerned with this story, and we encourage them to ask the NSA to explain exactly what happened. We think the government should be far more transparent about its vulnerabilities policy. A start would be releasing a current version of the VEP without redacting the decisionmaking process, the criteria considered, and the list of agencies that participate, as well as an accounting of how many vulnerabilities the government retains and for how long. After that, we urgently need to have a debate about the proper weighting of disclosure versus retention of vulnerabilities, and we should ensure that any policy that implements this decision is more than just a vague blog post or a document that lacks all “vigor.”

• 1. We have chosen not to directly link to the APL here for technical reasons. The Department of Defense uses its own Certificate Authority (CA) for authenticating websites, which is not trusted in browsers. We provide the url here as a convenience, but recommend strongly against adding additional CAs to your browser: https://aplits.disa.mil/processAPList.action