We announced Canary Watch a year ago as a coalition project to list Warrant Canaries and monitor them for changes or removal. Canary Watch was a joint project, with EFF, Freedom of the Press Foundation, NYU Law, Calyx and the Berkman Center.

Along the way, the project has been part of the massive popularization of the concept: we began with just eleven canaries listed, and now just a year later we have almost seventy. In the course of tracking those, we have learned many lessons about the different types of canaries that are present on the web, as well as what happens when a canary goes away.

In that way, the Canary Watch project has been a major success, and we’ve decided that it has achieved the goals we set out for it. As of today we will no longer accept submissions of new canaries or monitor the existing canaries for changes or take downs.

Transparency reports and warrant canaries have an important role to play in the fight against illegal and unconstitutional national security process, including National Security Letters and other secret court processes. We have not received any court orders or government requests to shut down the Canary Watch project. Rather, all of the members of the Canary Watch coalition have come to the agreement that the project has run its course and has come to a natural ending point.

Over the course of the project, we have learned some things about the nature of canaries on the Web which are important for anyone working with warrant canaries or doing activism around them.

The Number of Warrant Canaries Is Increasing

We started with eleven canaries; within four months that number grew to fifty; at the end of the project there were almost seventy warrant canaries in the Canary Watch database, with requests to add dozens more. In the last month the number of searches for warrant canaries grew by an order of magnitude. This is likely thanks in large part to the disappearance of reddit's warrant canary from their 2016 transparency report. The last year has, without a doubt, been a banner year for awareness of warrant canaries.

Search frequency for the term “Warrant canary” since 2007

Warrant Canaries Provide Interesting, But Not Definitive Information

Since July of 2013 Pinterest has been publishing a warrant canary which simply read "National Security: 0" as a part of their quarterly transparency report.  In 2015, Pinterest's number of national security requests changed from 0 to 0-249, reported for January to June, and July to December (instead of quarterly).  What prompted this move?  Under the law, a company that has received a national security request can report in bands of 250, starting at 0, semiannually. Thus, there is certainly the strong implication that Pinterest did receive a national security request, because it would have otherwise have continued to report 0.

Yet, in our time working with Canary Watch we have seen many canaries go away and come back, fail to be updated, or disappear altogether along with the website that was hosting it. Until the gag orders accompanying national security requests are struck down as unconstitutional, there is no way to know for certain whether a canary change is a true indicator. Instead the reader is forced to rely on speculation and circumstantial evidence to decide what the meaning of a missing or changed canary is.

Warrant Canaries Can Be Fickle

We also observed warrant canaries behaving in unexpected ways. Sometimes a canary would have subtle changes in language or grammar, which can be hard to interpret. Other canaries would regularly change what URL they were located at, and for others domains these URL changes were sudden and unexpected. Canaries often were not updated at all, or were updated several days or weeks late. Sometimes the warrant canary, along with the entire website would disappear without explanation or reason, and sometimes just the warrant canary would disappear and come back later, unchanged. All of this uncertainty caused numerous false alarms, which made it difficult to monitor warrant canaries. Additionally, this chaos served as a further demonstration of how difficult it is to interpret what it means when a warrant canary changes.

Warrant Canaries Come In Many Shapes and Sizes

One of the most surprising things that we have learned over the course of the Canary Watch project is that almost every canary is unique. We have seen canaries that were in PDFs, plaintext, HTML, and even images. We have seen canaries that were integrated into the website banner and canaries which were only available on Github. We have seen canaries that are signed using GPG, canaries that are part of a transparency report, canaries that include the day's weather and top news headlines. We have seen canaries that are updated on a daily basis and canaries which are updated once per year. We have seen canaries that were created once and then never updated again. Again, the fact that canaries are non-standard makes it difficult to automatically monitor them for changes or takedowns.

The major strides in our understanding about the nature and current status of warrant canaries and national security letters mean Canary Watch has definitely been a success. Moreover, it raised awareness and contributed to an important policy debate that is now well underway. In contrast to the uncertainty a year ago, it now seems that the Internet at large can offer robust and decentralized monitoring of warrant canaries; the rapid spread of the news when reddit’s canary disappeared is a testament to that fact.

Finally we would like to give a huge thank you to our coalition partners on this project for the last year: The Calyx Institute, Freedom of the Press Foundation, The Berkman Center, and the NYU School of Law.