There's an action movie cliché in which a cop inspects the body of a felled assassin or foot soldier and discovers a curious tattoo that ultimately leads to a rogue black-ops squadron, a secret religious sect, or an underground drug trafficking ring. 

The trope isn’t entirely Hollywood fantasy, but the reality of emerging tattoo recognition technology is closer to a dystopian tech thriller. Soon, we may see police departments using algorithms to scrape tattoos from surveillance video or cops in the field using mobile apps to analyze tattoos during stops. Depending on the tattoo, such technology could be used to instantly reveal personal information, such as your religious beliefs or political affiliations.

For years, law enforcement has used tattoos to identify criminal suspects as well as unidentified victims. Police have also used tattoos to map out subcultures and networks of gangs and hate groups. Until recently, however, tattoo matching and analysis has involved flipping through the pages of photo binders; any computer-assisted matching has been limited to metadata searches of keywords.

NIST's Official Tattoo Recognition Technology Logo

In 2014 and 2015, federal researchers at the National Institute of Standards and Technology (NIST) joined forces with the FBI to launch a program to accelerate tattoo recognition technology. As part of Tatt-C (the code name for NIST’s "Tattoo Recognition Technology Challenge"), officials assembled a giant dataset of prisoner tattoos and divvied it out to biometric companies, research institutions, and universities. They were asked to run five experiments to show how well their algorithms could match tattoos under various circumstances. 

Some tests involved matching different photos of the same person’s tattoo. Other experiments sought to match similar tattoos on different people based on their characteristics—such as a crucifix, Minnie Mouse, and Chinese calligraphy. These tests pose serious concerns for privacy, free expression, religious freedom, and the right of association.

Each one of these experiments correlated to a specific law enforcement use. The Tatt-C results, released last summer, now serve as a crystal ball into what law enforcement has planned for this technology over the years to come.

Here are the five tests and what they tell use about the future of tattoo recognition technology.

Related: Learn why EFF is calling for an end to this research. 

Note on the data: The Tatt-C dataset contained 15,000 images obtained by the FBI from prisoners. The dataset was split into subsets and sub-subsets for individual trials. Tatt-C participants self-reported their results, which were not independently verified. The percentages below reflect the accuracy within the experiment, and not necessarily how accurate the technology would perform in the real world.

Tattoo Detection: Why would police want algorithms that can detect whether an image has a tattoo or not? 

Any given law enforcement agency may be sitting on an immense, unsorted collection of images. Mugshots, scars, birthmarks, and tattoos—all mixed up together, some unlabeled, some mislabeled. Without computer assistance, it could take significant staff-power to sort through it all. NIST suggests that automated tattoo detection would streamline an agency’s ability to classify images.

Perhaps the more concerning use case for privacy advocates is that tattoo detection technology would also pave the way for algorithms to isolate tattoos from images scraped from the Internet or captured by security cameras.  

The bad news is the technology is already highly sophisticated. 

Tatt-C’s research team reported back that three different organizations’ algorithms could detect a tattoo in an image with more than 90% accuracy.  The private biometric technology company MorphoTrak (a subsidiary of Safran) claimed the best result; their algorithm was able to detect whether an image contained a tattoo or not with 96.3% accuracy.

Tattoo Identification: When we say biometrics, we are talking about unique physical or behavioral characteristics that can be used to identity you. Fingerprinting has been used by criminal justice agencies for over a century to identify suspects; tattoo recognition can be used in much the same way.

Let’s say a cop is questioning someone on the street who refuses to provide an ID card. The officer could run a photo of one of the person’s tattoos through a database to find a photo of the same tattoo captured during a previous arrest. One situation NIST imagines is applying tattoo recognition technology to video surveillance of a robbery in which the suspect is wearing a mask but a neck tattoo is visible. 

Just as facial recognition technology raises serious privacy concerns, people should be wary of tattoo recognition technology’s invasiveness. Not only can it identify everyday people caught on camera while going about their business, it could eventually lead to tracking people using their tattoos.

NIST asked Tatt-C participants to match a photo of a tattoo to other pictures of the photo taken over time.  Four different companies and research institutions reported that their algorithm could return a hit on the first result with more than 95% accuracy. Again, MorphoTrak came out on top, returning a hit with 99.4% accuracy.

Region of Interest: NIST uses “Region of Interest” to describe how well an algorithm can match a small piece of a tattoo to a wider image of the whole tattoo. For example, could the algorithm recognize that a tiny skull tattoo is part of a larger half-sleeve arm tattoo. 

The idea here is that sometimes only a portion of a tattoo is caught on surveillance; is that enough to identify someone if police have the whole tattoo on file? This technology would also help police match a tattoo, even if the person subsequently added more to the design. 

Yet again, MorphoTrak provided the most accurate results: the algorithm could return a hit on the first result with 94.6% accuracy. Purdue University, which has developed an app [.pdf] with support from the U.S. Department of Homeland Security, was close behind with 91.6% accuracy.

Mixed Media: When you get a tattoo, the artist rarely inks their first draft on your skin.  The artist will draw it out on paper, then turn it into a purple transfer to trace out with the needles. The question for researchers is whether an algorithm can reverse engineer this process; rather than matching tattoos to tattoos, can they match a tattoo to an image in another medium.

If a witness sees a tattoo during a crime, they could describe it for a sketch artist, who could run the sketch through a tattoo database.  Or, if an officer wants to see if a tattoo correlates to a gang symbol, the tattoo could be compared to street graffiti.

But this technology sets law enforcement on a dangerous path, since it would allow a police officer to learn more than just your identity, but your interests, political beliefs, or religion.  An investigator could plug an image of an Anarchist circle-A or the Republican elephant into its database to return a list of people who have tattoos of those images.  

This technology is on the horizon, but at this stage it is still relatively underdeveloped.

The MITRE Corporation—a non-profit organization that manages research centers on behalf of the federal government—produced the most successful results. The algorithm could produce matches within the first 10 results with 36.5% accuracy. 

Although that number is fairly low, that may not prevent law enforcement from using it to generate leads. However, less reliable algorithms have greater potential of capturing innocent people in investigations.

Tattoo Similarity:  One of the most worrisome applications of tattoo recognition technology is its potential ability to reveal connections or shared beliefs among a population. For example, rather than matching a particular tattoo of a crucifix with an individual, police could run the image of a crucifix through a database to produce a long set of people with similar cross tattoos. This essentially means police would be able to create lists of people based on their religion, politics, or other affiliations as expressed by their tattoos.

This type of tattoo matching could sweep up fans of the same bands or members of the same labor union or military unit. This application has a high likelihood of generating false positives—matching someone whose tattoo may be visually similar, but not actually symbolically similar. That could result in people being improperly associated with groups, such as gangs, with which they have no actual affiliation.

Law enforcement primarily wants to use this technology to identify members of gangs and hate groups, who often use coded symbols to express their affiliation. But that’s not necessarily what NIST researchers focused on during Tatt-C’s “Tattoo Similarity Experiments,” which tested how well algorithms could match different tattoos with similar visual features. Many of the images NIST asked participants to analyze were religious symbols—often Catholic iconography, such as hands holding rosaries and Jesus Christ’s crucifixion.

This should raise bright red flags for those concerned about religious freedom, especially in light of how authoritarian governments have used tattoos to oppress religious minorities. Nazi Germany’s use of tattoos to track Jews during the Holocaust comes to mind. Indeed, the six-pointed Star of David was one of the images used during the NIST experiments. However, in that case, the star also serves as the symbol of the Gangster Disciples, a Chicago street gang. So even when law enforcement is attempting to use tattoos to investigate gangs, people who are simply expressing their religion could be labeled as affiliates of criminal gangs.

The good news is that the technology is still in its early stages.  Researchers attributed the drop in accuracy to a problem they called the “semantic gap.” That refers to the difficulty computers have in divining meaning from tattoos that contain relevant symbols, but are not clearly visually similar. MITRE achieved the best results; its algorithm could establish a correct match within the first 10 results with 14.9% accuracy.

What’s Next

Although the NIST and FBI experiments were largely academic research exercises, law enforcement is already deploying the technology. Purdue University, with support from the Department of Homeland Security, has developed a graffiti and tattoo matching app—GARI—that is now in use by law enforcement agencies across the state of Indiana. Meanwhile, companies like MorphoTrak and DataWorks are now offering tattoo recognition as part of biometric software packages that also include fingerprint scanning, iris scanning, DNA analysis and facial recognition. We know that sheriff's departments in California have contracts with these companies.  

It’s also clear that the lessons learned from the Tatt-C project are being used to refine future research and tattoo recognition technology. Following the Tatt-C project, NIST released training materials for law enforcement that explained how camera framing and lighting can make tattoos more easily recognizable by algorithms. Researchers further recommended that an even larger dataset—more than 100,000 images—be compiled for distribution to third-party researchers. 

This summer, NIST plans to launch its next major series of experiments: Tatt-E, short for the Tattoo Recognition Technology Evaluation. Using tattoo databases from the Michigan State Police, Tennessee Department of Corrections, and the Pinellas County Sheriff’s Office in Florida, NIST intends to run a similar set of tests internally, connecting to each algorithm through an API. 

For the sake of civil liberties, privacy, and dignity, we believe that NIST should halt this program immediately. Take action now to call for an end to experimentation with our tattoos.