Facebook’s Reported ‘Rulebooks’ Show It’s Got A Serious Content Moderation Problem

Facebook’s Reported ‘Rulebooks’ Show It’s Got A Serious Content Moderation Problem

There are mere days left in 2018, but Facebook’s eternal year of reckoning continues.

A new report from the New York Times has pulled back the curtain on part of Facebook’s internal struggle to get its hands around the complex problems housed on its platform, not the least of which include disinformation and hate. The report comes as part of a monthslong investigation by Times’ Max Fisher, who obtained a massive stock of documents intended to guide thousands of moderators on the platform whose job it is to manage potentially problematic content. According to the times, Facebook’s so-described rulebooks, which the Times says contain “numerous gaps, biases and outright errors.”

The Times was reportedly provided the documents—some of which were previously reported by Motherboard—by “an employee who said he feared that the company was exercising too much power, with too little oversight—and making too many mistakes.” The report paints a portrait of haphazardly assembled rulebooks comprising loose spreadsheets and PowerPoints of rules and stipulations by which moderators are tasked with policing content. The documents, the Times says, can be confusing when taken as a whole:

One document sets out several rules just to determine when a word like “martyr” or “jihad” indicates pro-terrorism speech. Another describes when discussion of a barred group should be forbidden. Words like “brother” or “comrade” probably cross the line. So do any of a dozen emojis.

The guidelines for identifying hate speech, a problem that has bedeviled Facebook, run to 200 jargon-filled, head-spinning pages. Moderators must sort a post into one of three “tiers” of severity. They must bear in mind lists like the six “designated dehumanising comparisons,” among them comparing Jews to rats.

The Times reported that while the rulebooks’ architects consult with outside groups, they “are largely free to set policy however they wish.” The teams responsible for assembling the rulebooks are “mostly young engineers and lawyers” who attempt “to distill highly complex issues into simple yes-or-no rules,” The Times said. That undertaking reportedly proves difficult for moderators, some of whom the Times says rely on Google Translate and have “mere seconds to recall countless rules” while combing through up to a thousand posts daily.

But more troubling than Facebook’s arbitrary collection of rules intended to police its billion-plus users—posts by whom can run the gamut of tasteless memes to calculating and potentially dangerous political propaganda—is the significant political power it wields. In deciding who is allowed a platform on Facebook’s site, the report illustrates that can be incredibly tricky.

One example cited by the Times was a fuel violence against Muslims for years, which the Times said occurred in part because of a “paperwork error” in its rulebooks that instructed allowing posts that should have in fact been removed.

Much of the Times report fills in the blanks about procedures at Facebook that have long failed to manage the problems on its platform. But it also illustrates the extent to which Facebook is struggling to handle the issues that continue to arise as it attempts to comply with the demands of respective governments.

Try as it may to manage its own product, Facebook has a Facebook-sized problem that likely isn’t going away anytime soon.

Updated 12/27/18 9:15 p.m. ET: Updated to reflect that some documents reported by the New York Times on Thursday were previously reported by Motherboard.

[New York Times]