Since the opening kick, much of the FIFA World Cup chatter has taken place online. But not all banter has remained friendly. A lot has been written about content moderation, but with the World Cup now underway, the debate over what can or should be permitted online, and whose responsibility it is to determine the rules of engagement, is in the spotlight again.
Players missing penalties have seen reams of hate speech directed their way. French players of Afro descent have had online posters question their heritage and allegiance. And during the group stage, when a clip of Portuguese national team player João Neves talking about his colleague Cristiano Ronaldo was taken out of context and circulated across the internet, Neves found himself in the line of fire of diehard Ronaldo fans. Allegations of arrogance and disrespect quickly turned into ad hominem attacks and flooded his replies and comment sections. It didn’t stop there. Photos posted by Neves’ girlfriend weren’t spared from hateful comments and general misogyny. She limited comments on her Instagram posts and then shut them off completely. Pundits say that heated tournaments such as the Super Bowl or the World Cup are ripe for this type of discourse, and are especially so thanks to Generative AI, which makes it easier to create misinformed and even inaccurate content. Indeed, after Argentina’s controversial win against Egypt last week, AI-generated videos of Jimmy Kimmel calling the refereeing biased went viral online, though he had not spoken on the topic. This brings new challenges to Trust & Safety (T&S) teams, which are now more lean than ever.
Football (or, soccer for Americans) has always brought out the best and worst of its fans, inspiring the creation of important rules and norms during match time and online too. In one of the most famous cases, after Brazilian forward Vinícius Júnior was on the receiving end of a coordinated racist campaign with users replying to all his posts across social media with racist slurs and then monkey emojis to bypass filters, Meta changed their internal hate speech guidance and invested in improving hate speech classifiers to detect code-mixed, emoji-filled posts. Beyond industry, numerous academic studies have seen football fan communities as a place to source ideas and test new theories of content moderation. FIFA has even created dedicated channels, such as a Social Media Protective Service team, to monitor and report hateful comments directly to social media companies to improve detection and removal.
While there are plenty of well-intentioned efforts to combat inappropriate behavior both on and off the pitch, some efforts are more successful than others. This blog post highlights a few improvements that can be made to Trust & Safety practices to make sure fans and players aren’t on the receiving ends of targeted attacks, can stay up-to-date with their favorite teams, and still engage freely in friendly banter.
Overreliance on automated safety mechanisms undermine online safety
Since mass layoffs at tech companies in 2022 and subsequently since, which disproportionately targeted T&S teams, companies are now shorthanded in their efforts to ensure equal and robust application of their content moderation policies. It’s bad business for companies to not do some sort of content moderation. And regardless of whether their policies remain aligned with what users say matters to them, companies still make at least nominal references to eradicating hate on their platforms. However, when it comes to enforcing these policies, there’s much to be desired. Given companies’ intentional reduction in Trust & Safety capacity and expertise in-house, many are turning to automated tools to keep users safe online. However, research shows that these automated means are error-prone and not sophisticated enough to distinguish between harmful speech and joy. For example: an LLM used to flag derogatory speech can’t parse intent in a sentence like, “William is such a Cheesehead,” where the term could be used to refer to a World War Two-era pejorative against the Dutch, or to fans of the Green Bay Packers, whose fans are called cheeseheads, or a third thing entirely.
Loss of cultural, linguistic, and football expertise in T&S leaves key gaps uncovered
Context is critical to the effective automated application of content guidelines and rules, and to ensuring that this process correctly distinguishes between what is in good fun and what is harmful. It’s especially more important given the global scale of football fandom, which makes adjudications of what is harmful and what is not more complex and culture-specific. The word fresa, for example, could mean “strawberry” in Catalan, refer to a member of the elite in Mexico, or be used as a homophobic slur in parts of Latin America.
Now, this nuance and the contextual expertise required to parse it may be elusive within T&S teams. With repeated layoffs and chronic underinvestment in context-driven T&S, experts say companies are losing exactly the type of global expertise they need to ensure accuracy of their systems, particularly when dealing with more complex, sophisticated contexts. This expertise is more important when dealing with content from users outside the United States, who are more likely to code-mix, post using Standard American English keyboards (such as posting in Arabizi), or use multilingual slang when they speak. CDT research has shown how these types of speech are often missing in the majority of training datasets and evaluation resources that T&S engineers use to train and test automated content analysis systems. An analysis of leading content classification LLMs showed a varying degree of accuracy in detecting racist speech targeted at football players. One hypothesis is that this is because of the way users speak online, often in code-mixed or multilingual ways; in one social media content analysis, Microsoft researchers found that users often post in English when talking about their jobs, and swear in their mother tongue when they post about cricket or Bollywood.
Top-down approaches miss nuanced coverage
When it comes to football banter, what’s tolerated online may depend on where you’re from. In other words, your mileage may vary. This nuance is especially important when influential actors have different interpretations of what should be permitted or not. For example, Mexican football executives have previously condoned fans chanting slurs as a long-standing way to pressure the opponent; however, UK football executives have condemned it.
But, content moderation that applies universal rules onto a global set of users will invariably fall short when it comes to meeting each and every user’s, and fandom’s, set of preferences. Nimble and user-controlled content moderation solutions are more likely to help. This could look like user-controlled filtering levers or tools like Block Party, which enable users to see only some content and block others entirely. This could be helpful if, say, a user only wants to see posts by Norwegian viking striker Erling Haaland and no other player, or wants to ensure no content questioning Cristiano Ronaldo’s legacy takes up space on their feed.
What can we do?
One way T&S teams can ensure their automated tools work for their communities is by using independently-created evaluations that are tailored to testing tools’ performance in context and languages other than English. ML Commons, for instance, offers a robust repository of evaluations that represent the speech patterns of non-English language speakers and contexts. Their PRISM playbook also serves as an important path for internal teams to incorporate global expertise to shape and make systems more robust. Making more T&S tools open-source, as the Robust Online Safety Tooling (ROOST) initiative is doing, is another way T&S teams and engineers can work towards making tools more nimble, customizable, and auditable to fit the needs of live, fast-moving events and constantly evolving fandoms.
Because much of sports chatter happens across modes, including still images, edited videos, and livestreams, it’s also essential to invest in research on multimodal and multilingual vectors of hate and abusive speech. How those patterns affect users from the global majority, and how content moderation systems should adapt to address the impacts, is crucial to improving online experiences for all who play ball.