Recent attacks on encryption have diverged. On the one hand, we’ve seen Attorney General William Barr call for “lawful access” to encrypted communications, using arguments that have barely changed since the 1990’s. But we’ve also seen suggestions from a different set of actors for more purportedly “reasonable” interventions, particularly the use of client-side scanning to stop the transmission of contraband files, most often child exploitation imagery (CEI).

Sometimes called “endpoint filtering” or “local processing,” this privacy-invasive proposal works like this: every time you send a message, software that comes with your messaging app first checks it against a database of “hashes,” or unique digital fingerprints, usually of images or videos. If it finds a match, it may refuse to send your message, notify the recipient, or even forward it to a third party, possibly without your knowledge.

On their face, proposals to do client-side scanning seem to give us the best of all worlds: they preserve encryption, while also combating the spread of illegal and morally objectionable content. 

But unfortunately it’s not that simple. While it may technically maintain some properties of end-to-end encryption, client-side scanning would render the user privacy and security guarantees of encryption hollow. Most important, it’s impossible to build a client-side scanning system that can only be used for CEI. As a consequence, even a well-intentioned effort to build such a system will break key promises of the messenger’s encryption itself and open the door to broader abuses. This post is a technical deep dive into why that is.

A client-side scanning system cannot be limited to CEI through technical means

Imagine we want to add client-side scanning to WhatsApp. Before encrypting and sending an image, the system will need to somehow check it against a known list of CEI images.

The simplest possible way to implement this: local hash matching. In this situation, there’s a full CEI hash database inside every client device. The image that’s about to be sent is hashed using the same algorithm that hashed the known CEI images, then the client checks to see if that hash is inside this database. If the hash is in the database, the client will refuse to send the message (or forward it to law enforcement authorities).

At this point, this system contains a complete mechanism to block any image content. Now, anyone with the ability to add an item to the hash database can require the client to block any image of their choice. Since the database contains only hashes, and the hashes of CEI are indistinguishable from hashes of other images, code that was written for a CEI-scanning system cannot be limited to only CEI by technical means.

Furthermore, it will be difficult for users to audit whether the system has been expanded from its original CEI-scanning purpose to limit other images as well, even if the hash database is downloaded locally to client devices. Given that CEI is illegal to possess, the hashes in the database would not be reversible.

This means that a user cannot determine the contents of the database just by inspecting it, only by individually hashing every potential image to test for its inclusion—a prohibitively large task for most people. As a result, the contents of the database are effectively unauditable to journalists, academics, politicians, civil society, and anyone without access to the full set of images in the first place.

Client-side scanning breaks the promises of end-to-end encryption

Client-side scanning mechanisms will break the fundamental promise that encrypted messengers make to their users: the promise that no one but you and your intended recipients can read your messages or otherwise analyze their contents to infer what you are talking about. Let’s say that when the client-side scan finds a hash match, it sends a message off to the server to report that the user was trying to send a blocked image. But as we’ve already discussed, the server has the ability to put any hash in the database that it wants. 

Given that online content is known to follow long-tail distributions, a relatively small set of images comprises the bulk of images sent and received. So, with a comparatively small hash database, an external party could identify the images being sent in a comparatively large percentage of messages.

As a reminder, an end-to-end encrypted system is a system where the server cannot know the contents of a message, despite the client’s messages passing through it. When that same server has direct access to effectively decrypt a significant portion of messages, that’s not end-to-end encryption.

In practice, an automated reporting system is not the only way to break this encryption promise. Specifically, we’ve been loosely assuming thus far that the hash database would be loaded locally onto the device. But in reality, due to technical and policy constraints, the hash database would probably not be downloaded to the client at all. Instead, it would reside on the server.

This means that at some point, the hash of each image the client wants to send will be known by the server. Whether each hash is sent individually or a Bloom filter is applied, anything short of an ORAM-based system will have a privacy leakage directly to the server at this stage, even in systems that attempt to block, and not also report, images. In other words, barring state-of-the-art privacy-preserving remote image access techniques that have a provably high (and therefore impractical) efficiency cost, the server will learn the hashes of every image that the client tries to send.

Further arguments against client-side scanning

If this argument about image decryption isn’t sufficiently compelling, consider an analogous argument applied to the text of messages rather than attached images. A nearly identical system could be used to fully decrypt the text of messages. Why not check the hash of a particular message to see if it’s a chain letter, or misinformation? The setup is exactly the same, with the only change being that the input is text rather than an image. Now our general-purpose censorship and reporting system can detect people spreading misinformation… or literally any text that the system chooses to check against. Why not put the whole dictionary in there, and therefore be able to decrypt any word that users type (in a similar way to this 2015 paper)? If a client-side scanning system were applied to the text of messages, users would be similarly unable to tell that their messages were being secretly decrypted.

Regardless of what it’s scanning for, this entire mechanism is circumventable by using an alternative client to the officially distributed one, or by changing images and messages to escape the hash matching algorithm, which will no longer be secret once it’s performed locally on the client’s device.

These are just the tip of the iceberg of technical critiques, not to mention policy reasons, we shouldn’t build a censorship mechanism into a private, secure messenger.