Imagine trying to do online research on breast cancer, or William S. Burroughs’ famous novel Naked Lunch, only to find that your search results keep coming up blank. This is the confounding situation that faced Microsoft Bing users in the Middle East and North Africa for years, made especially confusing by the fact that if you tried the same searches on Google, it did offer results for these terms.

Problems caused by the voluntary blocking of certain terms by intermediaries are well-known; just last week, we wrote about how payment processors like Venmo are blocking payments from users who describe the payments using certain terms—like Isis, a common first name and name of a heavy metal band, in addition to its usage as an acronym for the Islamic State. Such keyword-based filtering algorithms will inevitably results in overblocking and false positives because of their disregard for the context in which the words are used.

Search engines also engage in this type of censorship—in 2010, I co-authored a paper [PDF] documenting how Microsoft Bing (brand new at the time) engaged in filtering of sex-related terms in the Middle East and North Africa, China, India, and several other locations by not allowing users to turn off “safe search”. Despite the paper and various advocacy efforts over the years, Microsoft refused to budge on this—until recently.

At RightsCon this year, I led a panel discussion about the censorship of sexuality online, covering a variety of topics from Facebook’s prudish ideas about the female body to the UK’s restrictions on “non-conventional” sex acts in pornography to Iceland’s various attempts to ban online pornography. During the panel, I also raised the issue of Microsoft’s long-term ban on sexual search terms in the Middle East, noting specifically that the company’s blanket ban on the entire region seemed more a result of bad market research than government interference, based on the fact that a majority of countries in the MENA region do not block pornography, let alone other sexual content.

Surprisingly, not long after the conference, I did a routine check of Bing and was pleased to discover that “Middle East” had disappeared from the search engine’s location settings, replaced with “Saudi Arabia.” The search terms are still restricted in Saudi Arabia (likely at the request of the government), but users in other countries across the diverse region are no longer subject to Microsoft’s safe search. Coincidence? It's hard to say; just as we didn't know Microsoft's motivations for blacklisting sexual terms to begin with, it was no more transparent about its change of heart.

Standing up against this kind of overbroad private censorship is important—companies shouldn’t be making decisions based on assumptions about a given market, and without transparency and accountability. Decisions to restrict content for a particular reason should be made only when legally required, and with the highest degree of transparency possible. We commend Microsoft for rectifying their error, and would like to see them continue to make their search filtering policies and practices more open and transparent.