This is a fair to mediocre guide to state-sponsored trolling, there is a strong bias in the writing and a distinct lack of any documentation. I apologize for not sharing this guide previously, but I was just sent the link this morning.
Just a quick analysis of their paragraph focusing on the United States.
“UNITED STATES The Institute for the Future cited the U.S. as an example of “state incited” trolling, “in which the government maintains an arm’s length distance from the attack,” but uses proxies to signal state support for it. For instance, it noted President Trump referring to journalists as “enem[ies] of the people” – a message that many supporters took up on social media.”
“State incited” trolling. Russia has called this “patriotic” in the context of hacking, but the label can equally apply to trolling. The way it is written implies the United States is actively encouraging trolls to work on their behalf, whereas this is not remotely the case. All the political parties have paid bots, trolls, and other information warriors, but this is not the case for the nation, writ large. I cannot say the CIA does not have such a program, it is very likely they do, but operating mostly outside the United States.
The second part where Trump refers to journalists as “enem[ies] of the people” is a label which brands many journalists as biased to the extreme, in favor of the left. The accusation is that journalists do not abide by journalistic standards, many are biased, not fair, not objective, neglect to verify or corroborate, and do not adhere to any sort of journalistic standard. This article contains an example in the above excerpt.
The distinct lack of documentation makes we wonder about the veracity of the rest of the article. 7 countries are referred to by the report from the “Institute for the Future”, How Governments Are Deploying Disinformation as Part of Broader Digital Harassment Campaigns. 18 countries are cited in this article, but only 12 countries have references outside the study. The worst is, by far, the paragrah about the United States, again, the excerpt is above. There is an accusation through implication which contains absolutely no documentation.
Many references to state-sponsored trolling programs contain an accusation but the proof is in the form of a fairly biased article which does not contain evidence.
A notable exception appears to be in Malta, an exceptional sidebar is in the article. While not direct evidence of a state-sponsored troll program, there appears to be no other conclusion.
Another sidebar on such a program in Venezuela includes the following damning sentence.
“The Ministry of Communications distributes memes, hashtags and trolling targets via dedicated channels on Telegram, an encrypted messaging app, according to Marianne Diaz, a Venezuelan researcher.”
As written, Venezuelan Ministry of Communications program appears to mirror the Russian information warfare program, where centrally established guidance sheets, or Temniki, are then distributed. I strongly encourage Marianne Diaz to safely explore this program. Obtaining a copy of their ‘guidance sheets’ with memes, hashtags, and targets should be carefully pursued while maintaining safety first. Unfortunately, Ms. Diaz Hernández is taking a three-month sabbatical from social media.
If someone were to take this report and really work to establish strong evidence of state-sponsored trolling, this could be a legitimately strong reference guide. As it stands I do not endorse establishing any decisions or even formulating an overall opinion based on this article. I do endorse their overall conclusion, that many countries do have a state-sponsored trolling program, but that is only based on years of observation and a strong gut feeling. I know political parties pay trolls to support their candidate and their positions, I know that from first-hand observations. Outside of Russia, China, and Iran, however, I have not witnessed any state-sponsored trolling programs. I have strong suspicions of these practices in several other countries, and it does make sense, but I have no evidence which I have seen nor can submit.
As for the United States, outside classified programs, state-sponsored trolling is unthinkable. It would be a PR nightmare if the program were ever discovered, so any attempt at establishing such a program is ridiculous.
July 19, 2018
Journalist Nedim Turfent was reporting on a brutal counterterrorism operation in Turkey’s Kurdish region when he published video of soldiers standing over villagers, who were face down with their hands bound. Soon, odd messages seeking Turfent’s whereabouts began appearing on his Facebook page.
Then, Twitter accounts linked to Turkish counterterrorism units joined in, taunting locals with a single question—“Where is Nedim Turfent?”—as soldiers torched and raided more villages.
The threat was clear: Give him up, or you’re the next target.
That was in the spring of 2016. Within days, Turfent was in the military’s hands, and he was eventually charged with membership in a terrorist organization. An anonymous Twitter account capped off the social media manhunt by tweeting a picture of Turfent in custody, handcuffed and haggard. Then soldiers doused the office of his employer, Dicle News Agency, with gasoline and set it ablaze. Turfent remains behind bars.
Tina Urso went to bed on April 21 pleased with the small protest she helped organize in London around the visit of Malta’s prime minister. She wanted to call attention to the country’s unusual practice of selling passports to foreigners and the money laundering it has engendered. By the time she woke up, her Facebook feed was deluged with threats of violence and misogynist insults, including the false charge that she ran an escort service. Researchers concluded the attacks were coordinated through private Facebook groups administered by government employees and officials of Malta’s ruling Labour Party. Participants would eventually publish her parents’ address, as well as her confidential National ID card number. “My Facebook account was flooded with notifications, people sharing everything about me, manipulating photos taken from my profile,” Urso said. “It was just insane what they were able to do in just a few hours.”
Only a few years after Twitter and Facebook were celebrated as the spark for democratic movements worldwide, states and their proxies are hatching new forms of digitally enabled suppression that were unthinkable before the age of the social media giants, according to evidence collected from computer sleuths, researchers and documents across more than a dozen countries.
Combining virtual hate mobs, surveillance, misinformation, anonymous threats, and the invasion of victims’ privacy, states and political parties around the globe have created an increasingly aggressive online playbook that is difficult for the platforms to detect or counter.
Some regimes use techniques like those Russia deployed to influence the 2016 U.S. presidential election, while others are riffing in homegrown ways. And an informal but burgeoning industry of bot brokers and trolls-for-hire has sprung up to assist. The efforts have succeeded in many cases, sending journalists into exile or effectively silencing online expression.
In Venezuela, prospective trolls sign up for Twitter and Instagram accounts at government-sanctioned kiosks in town squares and are rewarded for their participation with access to scarce food coupons, according to Venezuelan researcher Marianne Diaz of the group @DerechosDigitales. A self-described former troll in India says he was given a half-dozen Facebook accounts and eight cell phones after he joined a 300-person team that worked to intimidate opponents of Prime Minister Narendra Modi. And in Ecuador, contracting documents detail government payments to a public relations company that set up and ran a troll farm used to harass political opponents.
Many of those findings are contained in a report released this week by a global group of researchers that uncovered evidence of state-sponsored trolling in seven countries, and Bloomberg reporters documented additional examples in several others. The report is by the Institute for the Future, a non-partisan, foresight research and public policy group based in Palo Alto, California.
“These campaigns can take on the scale and speed of the modern internet,” the report said. “States are using the same tools they once perceived as a threat to deploy information technology as a means for power consolidation and social control, fueling disinformation operations and disseminating government propaganda at a greater scale than ever before.”
Almost two years in the making, the report grew out of an earlier project commissioned by Google but never published. Researchers for the company’s Jigsaw division, its technology incubator, documented vicious harassment campaigns that were intended to appear spontaneous but in fact had links to various governments. These campaigns often operate “under a high degree of centralized coordination and deploy bots and centrally-managed social media accounts designed to overwhelm victims and drown out their dissent,” according to an unpublished copy of the Google report obtained through an outside researcher.
In response to revolutions and social movements launched on Twitter and Facebook, national governments initially censored content, blocked access to social media and used surveillance technology to monitor their citizens. But it turned out to be far more effective to simply inundate the platforms with a torrent of disinformation and anonymized threats—what the researchers dubbed a strategy of “information abundance” made possible by the rapid spread of social media.
Turkey is a prime example, according to Camille Francois, who directed the Jigsaw project as a principal researcher at Google. Since the 2013 protests at Istanbul’s Gezi Park, President Recep Erdogan’s government has used a combination of online and offline repression to turn social media “into a near dead zone for genuine social protest in Turkey,” Francois said. “Five years later, there is very little organically organized activity.”
Social media companies such as Facebook and Twitter are struggling to counter the significant resources and ingenuity that national governments and others are investing to manipulate their platforms. Millions of fake accounts game the companies’ algorithms, manipulating what users see, while technologies such as location spoofing can make centrally controlled accounts appear to be posts by armies of real people.
Twitter reportedly suspended 70 million fake and malicious accounts in May and June. The company says it has taken preventive measures to address trolling and that any form of malicious automation is a violation of Twitter rules. Facebook announced this week that it will begin removing misinformation that serves to incite violence, and a spokesman said it has invested in more effective ways to fight fake accounts. “We enforce these policies whether the responsible party is acting individually, as part of a company, or acting on behalf of a government,” a Facebook spokesman said.
Experts say the companies need to do even more to confront the fact that they’re being used by some of the world’s most repressive regimes.
“People sometimes worry that Azerbaijan will shut down Facebook,” said Katy Pearce, a communications professor at the University of Washington who has studied the platform’s use in that country. “Why would it? Facebook is the most effective tool of control the government has.”
Rise of the Trolls
Social media is used to strike fear into opponents in both dictatorships and democracies. See how it varies from country to country.
Trolling for Faith and Modi
Sitting cross-legged on a charpoy, an Indian day bed, Mahaveer Prasad Khileri taps on his laptop, his face lit by the screen and a single bulb hanging from the ceiling of his dirt-walled house. He uses his computer and two smartphones to advocate on social media for an organization dedicated to the well-being of cows, which are sacred to Hindus. Khileri’s work serves as a penance of sorts for a time when his deep faith and social media skills found a more toxic expression. He’s a former troll for India’s ruling Bharatiya Janata Party, or the BJP. “At that time, poison was in my mind,” he said.
Khileri was recruited by two acquaintances into the party’s social media operation in February 2014, just as Modi was racing to become India’s next prime minister. He was given eight cell phones and ID’s for six different Facebook identities, he recalled in an interview in his home village of Jogaliya. He worked 18-hour days, toggling between legitimate campaign work and trolling of opponents and journalists, he said. When Modi won, the operation evolved as well, transitioning to a tool supporting Modi’s government.
Khileri worked in what the BJP calls its ‘IT Cell,’ which effectively operated as an ad hoc troll farm, he said. The development of the cell in the world’s largest democracy occurred around the same time that American authorities believe Russia began using such techniques to influence the 2016 presidential election. The researchers contributing to the institute and Google reports found similar timing in different countries and under various circumstances.
According to Khileri, the Indian version of the trolling toolkit included strategies meant to inflame sectarian differences, malign the Muslim minority and portray Modi as savior of the Hindus. Supervisors would set themes for the day and specify targets to attack. Khileri and 300 other paid trolls would create memes or cut-and-paste Twitter posts that were sent to WhatsApp groups of tens of thousands of party loyalists. Their reposts sent hashtags viral in minutes.
“Muslims slaughter cows, so we’d tell them, ‘When Modi comes, we will slaughter you,’” Khileri recalled. “We’d tell Hindus: ‘If you don’t vote for Modi, then Muslims will destroy you.’”
The former head of BJP’s IT CELL, Arvind Gupta, tweeted in December 2016 that neither the party nor the cell has ever encouraged trolling and that online support for the party comes from a voluntary, grass-roots movement. The current head of the cell, Amit Malviya, said he would comment only after seeing evidence that Khileri was a member of it. Khileri said he eventually quit the cell—which paid its members in cash and left no paper trail—after he grew increasingly uncomfortable with the power it amassed.
“At any time, they can control the situation of India. The troll army can call a nationwide strike, shut down the country,” he said. “They are able to push for fights between communities, to create communal tension or destroy communal harmony.”
From Jingles to Death Threats
Trolling efforts take different forms in different countries. Party youth groups were commandeered in some. Others developed highly structured volunteer armies. Some simply paid contractors to do the work. Always, though, organizers took pains to make the activity appear spontaneous.
The government assists people in opening social media accounts through “Candanga Points”—kiosks set up in town squares—part of an effort announced in 2017 to create digital militiamen, according to the Institute for the Future’s report. The Ministry of Communications distributes memes, hashtags and trolling targets via dedicated channels on Telegram, an encrypted messaging app, according to Marianne Diaz, a Venezuelan researcher. Among the targets have been Lorenzo Mendoza, a government critic and president of Empresas Polar, a food distribution company, that trolls sought to blame for chronic food shortages. Diaz said the kiosks are managed by the same local committees that determine who gets access to food coupons, effectively motivating people to troll for food in a devastated economy with rampant malnutrition.
In Ecuador, former President Rafael Correa engaged a Guayaquil-based public relations firm called Inteligencia Emocional to promote his 2012 reelection, complete with videos, jingles, Facebook fan pages and a unified social media front under the brand Yo Revolución.
But additional work by the firm was more secretive. It set up and maintained a network of pro-Correa social media accounts that harassed political opponents, according to documents released by Ecuador Transparente, a whistleblower website that opposed the Correa administration. In a proposal to the Ecuadorean government, the firm proposed to charge $15,000 per month to staff its network around the clock with “community managers” in a “war room,” enabling it to “totally neutralize” offending content on the internet.
Internal Inteligencia Emocional documents anonymously leaked to Ecuador Transparente described the operation as a “troll center” and said that, due to the sensitivity of the project, it would implement a series of security protocols. Those steps included encrypting data shared among computers, scrubbing metadata from all materials and obscuring the location of the computers.
Correa did not respond to requests for comment; nor did the head of Inteligencia Emocional, Kenneth Godwin.
Another set of leaked documents seen by Bloomberg showed that Ecuador’s trolling operation involved the country’s intelligence directorate. The operation eventually became notorious for death threats and trafficking in hacked personal material of journalists and political opponents. Even ordinary citizens who used social media to speak out against the government were targeted, according to Martha Roldós, an investigative journalist and the daughter of a former president of Ecuador who has been a target of the troll operations.
Organized Like Drug Cartels
In Mexico, the interference of fake Facebook and Twitter accounts in public discourse has been so pernicious during the tenure of outgoing President Enrique Pena Nieto that his political opponents have dubbed them “Penabots.” Bot networks have also been used against the government’s agenda.
Two years ago, investigative journalist and activist Alberto Escorcia began documenting one of Mexico’s unique contributions to trolling: loosely organized youth gangs that former members told Escorcia are paid to stage virtual hate-mob attacks on journalists or activists who confront the government.
With monikers such as the Holk Legion, the gangs are organized like the region’s infamous drug cartels, Escorcia said. In one case documented by Amnesty International, Holk Legion trolls used more than 2,000 accounts to send death threats to 10 journalists and public figures over two days. The blizzard of tweets came on the second anniversary of a mass kidnapping of students in Guerrero State, an event that has plagued Peña Nieto’s government amid evidence of police and military involvement. The trolls’ menacing tweets arrived as a counterweight to renewed public outrage sparked by the anniversary.
Eduardo Sanchez, a spokesman for the Pena Nieto administration, said that “there’s no such thing” when asked by Bloomberg about the allegations that some networks of trolls have promoted the government’s agenda.
The Google report that examined government-connected trolling was initially set to be published last year, according to researchers who collaborated with the company. Some of them said congressional and law enforcement investigations into Russia’s election meddling may have made the topic too sensitive.
A spokesman for the company’s Jigsaw division, Dan Keyserling, said not all research projects end up being published and that information from the report was used internally to improve tools that detect abusive content. Jigsaw supports the fact that other organizations are continuing research into state-sponsored trolling, he said.
How to Build a Botnet
Turkey’s experience shows how politicians turned social media platforms into tools of information control—and rapidly perfected their techniques. Erdogan’s government was badly shaken by protests that began in May 2013 and grew over weeks to focus on government corruption. Within months, the ruling party was fielding its own bot army, albeit a poorly disguised one. Researchers found a collection of nearly 18,000 pro-Erdogan Twitter accounts that used profile pictures taken from porn sites or public figures as American actress Megan Fox.
By 2017, the country’s digital troops had evolved into something more finer-tuned and threatening. A report by AccessNow, a digital rights organization, identified a collection of fake accounts posing as people sympathetic to a protest march scheduled for that summer. The event would focus attention on the crackdown on journalists, teachers and others following an attempted coup in 2016.
Martha Roldos, the daughter of the nation’s first democratically elected president, is no stranger to political mysteries: Almost four decades after her father died in a 1981 plane crash, Ecuadoreans still speculate about who was responsible. On Jan. 6, 2014, Roldos herself was targeted by what was then a new and mysterious form of attack. Her private emails were stolen by hackers, then published on the front page of a pro-government newspaper. Those communications were spread by government-controlled social media accounts, which also smeared and threatened her. Roldos, who runs an investigative journalism project in Ecuador called Mil Hojas, eventually helped piece together evidence showing that the government of President Rafael Correa had built and funded the secret troll operation that attacked her and also targeted hundreds of others.
One such account, with a picture of a young woman claiming to be a march supporter, posted a link to a website for participants. Visitors to the site had their devices infected with advanced malware that spied on their communications and tracked their movements, the AccessNow report found. The company that created the malware sells only to governments.
Researchers and journalists have documented similar bot armies in Argentina, Thailand, Bahrain and Saudi Arabia.
Saudi trolling was on display after the May 15 arrest of seven prominent Saudi women’s rights activists who had been demanding an end to the ban on female drivers. The Saudi government initially gave no information about the activists’ whereabouts. Concerned Twitter users posted messages followed by an Arabic hashtag “Where are the activists?” That was quickly countered with a hashtag circulated by a state-backed news organization labeling the activists “Agents of the Embassies,” alongside pictures of them with the Arabic word for “Traitor” stamped over their faces
It got uglier. “How’s prison food?” one account asked Loujain Al-Hathloul, a Western-educated 28-year-old who’d previously been detained after live-Tweeting herself driving. “You act like you’re something when you’re nothing,” read another, next to a knife emoji. Another account linked to an archive of Al-Hathloul’s tweets, so opponents could scour the words of an “enemy of the homeland.”
An analysis by Graphika, a social media intelligence firm based in New York, showed that the “Agents” hashtag was “pushed” in a highly coordinated way, said John Kelly, the company’s CEO and founder. Marc Owen Jones, a researcher and lecturer on Gulf affairs at Exeter University, found evidence that the hashtag was pushed by automated accounts tied to a vast pro-Saudi government botnet that he’d previously identified.
“It still remains to be seen whether this is a state-sponsored operation, the work of a PR company, or a wealthy individual’s unilateral project,” Owen Jones wrote about that botnet in February.
Misogyny and Rape Threats
Some of the most virulent attacks are aimed at women. The Institute for the Future report found that every identified instance of state-sponsored trolling involving female victims used heavily misogynistic language, including threats of rape and mutilation.
The graphic, relentless posts often prove to be powerful weapons, publicly singling out a regime’s opponents and legitimizing those who attack them, while leading some journalists to self-censor their reporting, according to the report.
Before she was killed by a car bomb last October 2017, Daphne Caruana Galizia, who gained fame writing about corruption on the tiny Mediterranean island nation of Malta, grew so anxious about the online attacks that greeted her blog posts that she often hesitated before publishing a new one, said her son, Paul Caruana Galizia.
Many of the violent comments against his mother were coordinated inside a handful of private Facebook groups run by government employees and which count among their members ruling Labour Party lawmakers. The secret groups, whose operations were revealed in the months following her death, circulated “thousands of really horrifically abusive comments” against his mother, he said.
One meme, which circulated shortly before her death, substituted her image for that of Jesus Christ next to Pontius Pilate, who asks the crowd: “What do you want me to do with her?” Comments encouraging violence poured in. “The dehumanization by the time she was killed around that period was complete,” her son said.
According to examples uncovered by researchers, trolls also threatened to sexually abuse Turkish journalist Ceydan Karan with a broken bottle, while Mexican scholar Rossana Reguillo was sent pictures of burned bodies with the warning, “This could happen to you.”
The fact that the threats are designed to look like they are coming from crowds of anonymous social media users makes them more ominous in some cases than if they had been issued directly by the government.
“That’s the genius of these types of attacks,” said Carly Nyst, one of the authors of the Institute for the Future report and an expert on the intersection of human rights and technology. “It’s hard to distinguish between what’s being manufactured on purpose and what is a popular uprising of opinion against the target.”
One question left unanswered by the reports is why so many nations developed strikingly similar trolling operations at around the same time. There is some evidence of information-sharing among countries, consultants and government functionaries.
Correa, the former president of Ecuador, now hosts a television show for the Spanish-language version of RT, the Russian news channel that is closely tied to sophisticated government-sponsored information campaigns. And last year, when Philippines President Rodrigo Duterte was in the Kremlin meeting with Russian President Vladimir Putin, a representative of the Philippines state-run media visited the Moscow headquarters of RT and entered into a partnership with the news agency and Russia’s communication ministry.
Duterte has used Facebook as a platform to aggressively target critics, and he has appointed well-known trolls to government posts.
In Azerbaijan, the government maintains a close relationship with Russia, which has exported some of its technological know-how, including bots, according to Pearce, the University of Washington professor.
Still, tracing the fingerprints of state-sponsored trolling remains a difficult task, largely because states go to great lengths to cover tracks. Francois, the former Google researcher, says the architecture of a trolling campaign is often akin to a hand inside a glove that organizes an action and then is removed.
While the glove is left behind for the world to see, the hand simply disappears.
With assistance from Sarah Frier, Carlos Manuel Rodriguez, Stephan Kueffner and Andrew Rosati.
Edited by Flynn McRoberts and John Voskuhl
Design and code Steph Davidson with help from James Singleton