Disinformation

According to the Dangerous Speech Project, “dangerous speech” is any form of expression (speech, text, or images) that can increase the risk that its audience will condone or participate in violence against members of another group.” This concept provides a constructive framework for thinking about hate speech that is liable to cause violence. Hallmarks of dangerous speech include dehumanization (referring to people as insects, bacteria, etc.) and telling people they face a mortal threat from a disfavored minority group.

The term “fake news” has no accepted definition and is inaccurately used as a synonym for disinformation. The term has been popularized in recent years and often is used to discredit information that one finds unfavorable, regardless of the truthfulness. As such, the terms “misinformation,” “disinformation,” or “mal-information” should be used in place of “fake news.”

Astroturfing: The attempt to create an impression of widespread grassroots support or interest in a policy or idea by using fake online accounts, such as networks of bots or fake pressure groups.

Bots: A bot is a software program that performs automated, repetitive tasks. In the case of social media, a bot can refer to an automated social media account.

Click Bait: A sensationalized or misleading article title, link, or thumbnail designed to entice users to view content.

Cyberviolence and Cyberbullying: Cyberviolence refers to acts of abuse using digital media. Cyberbullying refers to recurrent cyberviolence that is characterized by an imbalanced power dynamic.

Deepfake: Photos or videos that have been altered or entirely fabricated through machine learning to create a false depiction of something, such as a politician making an imagined statement. Using Deepfakes to confuse voters during election periods by fabricating statements from candidates and election officials is a prime example of information manipulation.

Doxxing: Publishing private or identifying information, especially sensitive personal information, about a person online with malicious intent.

Information Warfare: The use of ICT such as social media to influence or weaken an opponent, including through disinformation and propaganda.

Trolling: Creating intentional discord in an online discussion space, starting quarrels between users, or generally upsetting people by posting inflammatory, insulting, or off-topic messages.

Upload Filter: Automated computer programs that scan content uploaded to an online platform before it is published. Upload filters are used by social media platforms to identify content that violates the companies’ Terms of Service, such as child sexual abuse material (CSAM).

User Generated Content (UGC): Any content such as images, videos, text, and audio that has been posted by users on online platforms.

Virality: The tendency of an image, video, or piece of information to be circulated rapidly and widely on online platforms. When content goes “viral,” important context around the information being shared such as the source, author, and publication date is often lost (known as “context collapse”), potentially distorting how the information is received and interpreted by viewers.

For additional definitions, see the EU Disinfo Lab’s disinformation glossary, a collection of 150+ terms to understand the information disorder.

The rampant spread of mis- and disinformation, hate speech, and harassment on social media platforms has placed a spotlight on their content moderation policies and procedures, which are often implemented with little to no oversight. Social media platforms have been criticized for removing too much content, for not removing enough content, for developing algorithms that fail to detect the nuances of hate speech and misinformation, and for employing human moderators that suffer from low pay, poor working conditions, and trauma induced by the harmful content they review. Amid a cacophony of voices with opinions on what content should and should not be allowed on platforms, social media companies struggle to find the right balance between absolute free speech and protecting users from information manipulation. Legislation like the European Union’s Digital Services Act provides lawmakers with a tool to hold platforms accountable for their role in facilitating information manipulation and presents “a new way of thinking about content moderation that is especially valuable for the counter-disinformation community.”

See the Social Media resource^[8] for more information on content moderation.

Digital advertising has enabled the self-financing of websites and blogs that share hateful, incendiary, and/or misleading content. Programmatic advertising, which relies on automated technology and algorithmic tools to buy and sell ads, is particularly problematic because advertisers may end up financing these outlets without ever being aware their ads have been placed on them.

The online movement Sleeping Giants emerged to tackle this challenge by alerting companies when their ads are placed next to inflammatory or controversial content. Check My Ads is another resource that offers services to help prevent brands from being associated with disinformation and dangerous speech. The Ads for News initiative supports local journalism by providing brands and media buyers with a curated, global inclusion list of trusted local news websites that have been screened to exclude disinformation and other content unsuitable for brands.

See the Social Media resource^[9] for more information on digital advertising.

Content intended to discourage or prevent citizens from voting can take the form of inaccurate information about the date of an election, or efforts to persuade people that an election is rigged and their vote won’t matter. Voter suppression content that questions the legitimacy of electoral processes or the security of voting systems can also lay the groundwork for disputing election results, as information manipulation degrades trust in election management bodies. This content does not require malicious intent, as someone who unknowingly shares incorrect information about the deadline for voter registration, for example, could still confuse other citizens and prevent them from being able to vote in the election.

Different actors have different goals for influencing narratives during an election. For example, political parties and campaigns may use mis- or disinformation, mal-information, or propaganda to discredit the opposition or manipulate political discourse in a way that serves their campaign agenda, while a foreign adversary might seek to influence the election outcome, advance national interests, or sow chaos. A coordinated information manipulation campaign on Whatsapp—which included doctored photos, manipulated audio clips, and fake “fact-checks” discrediting authentic news stories—played a troubling role in boosting far-right candidate Jair Bolsonaro to the Brazilian presidency in 2018 (see the case studies  section for more on the role of information manipulation in Brazil).

Ultimately, information manipulation campaigns can destabilize political environments, exacerbate potentials for electoral-related violence, pervert the will of voters, entrench authoritarians, and undermine confidence in democratic systems more broadly. In many fragile democracies, strong democratic institutions that could help counter the impact of fake news and broader information manipulation campaigns, such as a robust independent media, agile political parties and sophisticated civil society organizations, remain nascent.

One tool that journalists, researchers, and civil society can use to counter information manipulation is fact-checking, or the process of verifying information and providing accurate, unbiased analysis of a claim. Hundreds of civil society fact-checking initiatives have sprung up in recent years around specific flashpoints, with the lessons learned and infrastructure built around those flashpoints then being applied to other issues that impact the same information ecosystems. During the 2018 Mexican general elections, the CSO-driven initiative Verificado 2018 partnered with Pop-Up News, Animal Político, AJ+ Español, and 80 other partners to fact-check and distribute election-related information, particularly among youth. Other successful fact-checking initiatives around the world include Africa Check, the Cyber News Verification Lab in Hong Kong, BOOM in India, Checazap in Brazil, the Centre for Democracy and Development’s Fact Check archive in West Africa, and Meedan’s Check initiative in Ukraine.

Social media companies have also invested in fact-checking teams and technologies, and have built partnerships with news agencies to counter the hoaxes, conspiracy theories, rumors, and propaganda that circulate on their platforms.

Fact-checks or “debunks” are effective, but they tend to be labor intensive and they are not read by everyone. “Prebunking” is an alternative approach based on the idea of inoculating people against false or misleading information by showing them examples of information manipulation so they are better equipped to spot and question it in the future. According to First Draft, there are three main types of prebunks:

1. Fact-based: Correcting a specific false claim or narrative
2. Logic-based: Explaining tactics used to manipulate
3. Source-based: Pointing out bad sources of information

In 2022, Google launched a prebunking campaign in Poland, the Czech Republic, and Slovakia that used videos to dissect different techniques seen in false claims about Ukrainian refugees. Researchers have also created online games that let players pretend to be trolls spreading fake news with the goal of improving people’s understanding of how information manipulation campaigns are built, and ultimately increasing their skepticism of them.

Media literacy programs can help increase citizens’ ability to differentiate between factual and false or misleading content by encouraging the use of critical thinking skills while consuming traditional and online media content. These programs can also raise awareness about how information manipulation disproportionately harms women and marginalized groups. Media literacy is a life-long process, and should be but one part of a comprehensive toolkit to counter information manipulation.

IREX’s Learn to Discern (L2D) initiative aims to build communities’ resilience to disinformation, propaganda, and hate speech in traditional and online media. After piloting a media literacy curriculum in classrooms, libraries, and community centers in Ukraine, L2D was extended to other countries including Serbia, Tunisia, Jordan, and Indonesia. Global initiatives such as the Mozilla Foundation’s Web Literacy framework and Meta’s digital literacy library provide access to educational media literacy materials and offer an opportunity for users to learn how to effectively navigate the virtual world.

Platforms can leverage design features to help mitigate information manipulation. For example, some platforms strategically send articles with corrective information to users who share false content. During the COVID-19 pandemic, Facebook showed users who engaged with false content messages that debunked the claims, and also redirected users who shared false information to authoritative sources like the World Health Organization (WHO) or the Centers for Disease Control and Prevention (CDC). These interventions, along with policies and features designed to limit the forwarding of viral content, aim to lessen the impact of information manipulation.

Open source intelligence is a method of gathering data and information from publicly available sources, including social media, websites, and news articles. Transparent, volunteer-led, crowdsourced information gathering and analysis can contribute to the debunking of falsehoods and myths, including in contexts like the Russia-Ukraine war.  The investigative journalism group Bellingcat has established itself as a leader in leveraging OSINT to report on conflicts and human-rights abuses, like the use of chemical weapons in Syria’s civil war.

Counterspeech generally includes any direct response to hateful or harmful language that seeks to undermine it. According to the Dangerous Speech Project, there are two types of counterspeech: organized counter-messaging campaigns and spontaneous, organic responses. Counterspeech can sometimes aim to educate the perpetrator of the harmful content, but more often it aims to reframe the online discussion for onlookers, change the tone in online public spaces, and crowd out harmful content with positive, inclusive messaging. One example of counterspeech in action is #jagärhär (#Iamhere), a Facebook group of about 75,000 people mostly based in Sweden that mobilized to add positive notes on comment sections where hatred and misinformation were being spread.

Reality Built on Lies: 100 Days of Russia’s War of Aggression in Ukraine

“Perhaps the most significant characteristic of the Kremlin’s disinformation campaign and information manipulation targeting Ukraine throughout [the] Russian war of aggression is its adaptability to new realities. In other words, over the past hundred days, the Kremlin has been constantly moving the goalposts of its disinformation in an effort to redefine what the objectives of the ‘special operation’ are, and what ‘success’ might look like for the Russian Armed Forces invading Ukraine… Besides moving the goalposts for success, Russian state-controlled disinformation outlets also actively advance false ‘humanitarianism’ narratives for another purpose. As the world inevitably learned of the senseless atrocities Russia committed in Ukraine, increasingly amounting to war crimes, the pro-Kremlin disinformation ecosystem went into overdrive to deny, confuse, distract, dismay, and shift the blame.”

Monetizing Misogyny

The global initiative #ShePersisted interviewed over one hundred women political leaders and activists all over the world in an attempt to understand the patterns, impact, and modus operandi of gendered disinformation campaigns against women in politics. Case studies on Brazil, Hungary, India, Italy, and Tunisia explore how gendered disinformation has been used by political movements, and at times the government itself, to undermine women’s political participation, and to weaken democratic institutions and human rights. Crucially, the research also looks at the responsibilities and responses that both state actors and digital platforms have taken—or most often, failed to take—to address this issue.

Disinfodemic: Deciphering COVID-19 Disinformation

“In contaminating public understanding of different aspects of the pandemic and its effects, COVID-19 disinformation has harnessed a wide range of formats. Many have been honed in the context of anti-vaccination campaigns and political disinformation. They frequently smuggle falsehoods into people’s consciousness by focusing on beliefs rather than reason, and feelings instead of deduction. They rely on prejudices, polarization and identity politics, as well as credulity, cynicism and individuals’ search for simple sense-making in the face of great complexity and change. The contamination spreads in text, images, video and sound.”

Is China Succeeding at Shaping Global Narratives about COVID-19?

“Faced with criticism over its handling of the pandemic, the Chinese government and its proxies have leveraged social media—especially Twitter—to spread its narratives and propaganda abroad… In early 2021, Chinese media spread claims that the Pfizer and Moderna vaccines are risky and even deadly, highlighting extremely rare sudden deaths or illnesses from people who received the vaccine in France, Germany, Mexico, Norway and Portugal… Taiwan was another major target of Chinese Covid-19 disinformation tactics—an unsurprising development given Beijing’s persistent use of disinformation against the island. China repeatedly sought to cast doubt on Taipei’s success at curtailing the spread of the virus… In addition to criticizing and spreading disinformation about other countries’ handling of the pandemic, Chinese media outlets and diplomats amplified unfounded conspiracy theories that SARS-CoV-2 originated outside of China.”

Los Eco-Ilógicos

Green Latinos, with support from Friends of the Earth, commissioned Graphika to study how false and misleading narratives about climate change reach Spanish-speaking communities online. The analysis aimed to understand how these narratives spread through the online ecosystem  of Spanish-speaking internet users, the groups and individuals who seed and disseminate them, and the tactics these actors employ. Through the analysis, Graphika identified a sprawling online network of users across Latin America and Spain that consistently amplify climate misinformation narratives in Spanish. While some of these accounts focus specifically on climate-related conversations, the majority promote ideologically right-wing narratives, some of which touch on climate change. Many of the narratives identified also overlapped with existing online conversations unrelated to climate change, such as COVID-19 misinformation or conspiracy theories about a secret ruling organization of totalitarian, global elites.

Fact from Fiction: Curbing Mis/disinformation in African Elections

“Election–related misinformation and disinformation seeks to, primarily, manipulate the decision-making of electorates, cast doubt in the electoral process, and delegitimize the outcome of the elections. This is a dangerous trend, particularly in fragile democracies where this is capable of inciting hate and stirring violent outbreak. Misinformation and disinformation were major factors in the escalation of post-election violence in Kenya following the 2017 general elections. Similarly, in 2020, the Central African Republic experienced deadly post-election violence as a result of a contested election, targeted disinformation efforts, and divisive language. The same happened in Cote d’Ivoire, where 50 people were killed in the political and intercommunal violence that plagued the presidential election on October 31, 2020.”

For additional context, see also the Atlantic Council’s June 2023 report on The Disinformation Landscape in West African and Beyond.