Leaked Facebook DIO ‘Blacklist’ Appears to Show 'Disproportionate Censoring’

Facebook, which has faced pressure to be more transparent about its policy against "dangerous individuals and organisations", has consistently refused to publish its so-called “Blacklist”, claiming it would endanger employees and permit banned entities to circumvent the policy.

A “blacklist” of over 4,000 people and groups that Facebook's Dangerous Individuals and Organizations (DIO) policy is built on, leaked by The Intercept, shows how the company may be meting out disproportionate censoring to various communities.

A reproduction of the DIO list is also accompanied by an associated policy document intended to aid the work of moderators who are tasked with deciding what posts to delete and which users to punish.

The "blacklist is divided into the categories Hate, Crime, Terrorism, Militarized Social Movements, and Violent Non-State Actors, all of them accordingly organised into a system of three tiers indicating the type of enforcement the company will take in regard to content". While no one featured on the DIO list is allowed onto the Facebook platforms, the tiers determine what other users are allowed to say about the banned entities.

The most restrictive is Tier 1, featuring alleged terrorist groups, hate groups and criminal organisations. Terror is defined as “organising or advocating for violence against civilians”, and hate as “repeatedly dehumanising or advocating for harm against” people.

A man is silhouetted against a video screen with a Facebook logo

© REUTERS / Dado Ruvic

Users may not express anything deemed to be praise or support for groups and people in this tier. In the terrorism category - which is said to constitute 70 percent of Tier 1 - nearly 1,000 of the entries note a “designation source” of “SDGT,” or Specially Designated Global Terrorists, a Treasury Department sanctions list. The latter was created by George W. Bush in the aftermath of the 11 September 2001 terror attacks.

A view of One World Trade Center from the North Pool, which marks the former site of the North Tower of the World Trade Center, at Ground Zero the night before the 15th anniversary of the September 11, 2001 terrorist attacks in the United States in New York

© AFP 2023 / Brendan Smialowski

Also sourced for listees in this tier were the Terrorism Research & Analysis Consortium private subscription-based database of purported violent extremists, and private terror-tracking Intelligence Group SITE.

Tier 1’s criminal category is almost entirely comprised of US street gangs and Latin American drug cartels, predominantly Black and Latino.

Tier 2 is labelled “Violent Non-State Actors”. To a greater degree it consists of groups like armed rebels who engage in violence targeting governments, with users allowed to praise groups in this tier for their nonviolent actions but not permitted to express any “substantive support” for the groups themselves.

Tier 3, which includes Militarized Social Movements that are predominantly white, is for groups that are not violent but repeatedly engage in hate speech, appear to be likely to become violent soon, or repeatedly violate the DIO policies. Other Facebook users are allowed to freely discuss those listed in Tier 3.

Moderator Guidelines

Internal Facebook materials also revealed by the outlet shed light on how the outsourced content moderation workforce of contractors is guided on censoring speech about the blacklisted people and groups.

While Facebook has previously offered a public set of guidelines, there were limited examples of what the terms specifically mean. The leaked materials aim to help moderators define user “praise” or “support” for a person or organisation listed in the DIO. Furthermore, detailed examples are suggested as being more illustrative, including a list of hypothetical situations to better determine the best action to pursue regarding flagged content.

Internal Facebook guidelines say users are allowed to apply neutral or critical speech regarding Tier 1 entities, to rule out any commentary be deemed positive and, accordingly, be construed as “praise.” Users are prohibited from saying anything that “seeks to make others think more positively” or “legitimise” a Tier 1 listee.

“Statements presented in the form of a fact about the entity’s motives” are acceptable, but anything that “glorifies the entity through the use of approving adjectives, phrases, imagery, etc.” is not, say the internal guidelines.

However, Facebook moderators are allowed to decide for themselves what constitutes dangerous “glorification” versus permitted “neutral speech”.

Internal Facebook guidelines say users are allowed to apply neutral or critical speech regarding Tier 1 entities, to rule out any commentary be deemed positive and , accordingly, be construed as “praise.” Users are prohibited from saying anything that “seeks to make others think more positively” or “legitimise” a Tier 1 listee.

“Statements presented in the form of a fact about the entity’s motives” are acceptable, but anything that “glorifies the entity through the use of approving adjectives, phrases, imagery, etc.” is not, say the internal guidelines.

For example, speaking “positively” about a designated entity, would be saying that the Sinaloa cartel donates much of its profits to charity. However, stating that an entity is not a threat relevant or worthy of attention is not deemed by the guidelines to be speaking “positively” about them. The material cite as an example in this case, “White Supremacy is not a threat”.

As another example, the materials offer the phrase, “Hitler did nothing wrong”. In this case, “legitimising” in reference to a designated entity is defined as making claims about moral, legal, other justification for criminal, hateful or terrorist activity. Claim of justification present renders it as a “violating” statement.

Regarding violent incitement, the guidelines permit, in the policy materials, calls for violence against “locations no smaller than a village”. Cited as an example is the statement “We should invade Libya.”

Challenging historical events and facts also does not violate Facebook guidelines, seemingly allowing for neutral debate.

A Facebook moderator working outside the US was cited by the publication as acknowledging that analysts “typically struggle to recognise political speech” and “struggle” to determine what content meets Facebook’s definitions of banned speech.

Facebook had previously refused requests by legal scholars and civil libertarians to publish the DIO “blacklist” list, despite the fact the company’s Oversight Board formally recommended publishing the list on multiple occasions, insisting the information is in the public interest.

‘Prioritizing Security, Limiting Legal Risks’

The company has repeatedly claimed that revealing the list would endanger employees and permit banned entities to circumvent the DIO policy. Facebook’s policy director for counterterrorism and dangerous organisations, Brian Fishman, responded to The Intercept with a written statement that emphasised that the “blacklist” is kept secret because “[t]his is an adversarial space, so we try to be as transparent as possible, while also prioritising security, limiting legal risks and preventing opportunities for groups to get around our rules.”

First, Facebook does not want violence organized or facilitated on its platform and the DOI list is an effort to keep highly risky groups from doing that. It’s not perfect, but that’s why it exists. 2/n

— Brian Fishman (@brianfishman) October 12, 2021

A Facebook spokesperson was cited as dismissing perceived concerns that the company gave some communities in the US special treatment, stating:

“Where American groups satisfy our definition of a terrorist group, they are designated as terrorist organisations (e.g. The Base, Atomwaffen Division, National Socialist Order). Where they satisfy our definition of hate groups, they are designated as hate organisations (For example, Proud Boys, Rise Above Movement, Patriot Front).”

Foreseeing criticism of disproportionate treatment along racial and and/or religious lines across the spectrum of Facebook’s tiers, the spokesperson underscored the presence of white supremacists and hate groups in Tier 1, adding that “focusing solely on” terrorist groups in Tier 1 “is misleading”.

“It’s worth noting that our approach to white supremacist hate groups and terrorist organisation is far more aggressive than any government’s... Our definition of terrorism is public, detailed and was developed with significant input from outside experts and academics. Unlike some other definitions of terrorism, our definition is agnostic to religion, region, political outlook, or ideology," said a Facebook spokesperson.