On the recent Waking Up podcast by Sam Harris, Renée DiResta talks about “The Information War“. She tells how she started researching the spread of disinformation on social media by looking into what “anti vaxxers” were doing to promote their point of view very successfully online. Viewed from a social media perspective their view seemed dominant, which was quite the opposite of what most people thought outside of social media. How did they do that? She talks about the techniques they used to game the system and distort reality, by making their minority view seem popular. Also, she found that once a person joins such a group they are targeted by other conspiracy groups. In Fast Company, Renée wrote “How Social Network Algorithms Are Distorting Reality By Boosting Conspiracy Theories ” where she describes how fairly simple algorithms intended to promote the most popular content have the unintended consequence of helping make crazy conspiracy theories go viral.
Once a user joins a single group on Facebook, the social network will suggest dozens of others on that topic, as well as groups focused on tangential topics that people with similar profiles also joined. That is smart business. However, with unchecked content, it means that once people join a single conspiracy-minded group, they are algorithmically routed to a plethora of others. Join an anti-vaccine group, and your suggestions will include anti-GMO, chemtrail watch, flat Earther (yes, really), and “curing cancer naturally” groups. Rather than pulling a user out of the rabbit hole, the recommendation engine pushes them further in.
She found activist groups on the left and right who were gaming social media platforms to spread their own narratives. It’s a new kind of activism or pseudo-activism. Maybe it’s fracking activism, which by going underground and cracking into people’s belief systems, you can manipulate them, change their minds without them seeing it happening and then get something you really want from them. If activism is meant to encourage you to act on a belief, this new pseudo-activism is intended to create disbelief and division by hacking into own belief systems. Like a fool, you no longer know what you actually believe and you do not act at all.
I first met Renée when she was working at O’Reilly Alpha Tech Ventures (OATV). She was an associate, a VC in training, which is perhaps where her Twitter handle “@noupside” comes from. She is remarkably smart and independent-minded. At that time, she was interested in the early formation of the maker community and helped organize along with Nick Pinkston an early hardware unconference in OATV’s offices in SF. She was co-author of a book called The Hardware Startup.
Renée started down a whole new path, as she describes in the podcast, once she became a mother. While researching kindergartens for her child, she thought to look into how many unvaccinated children might be attending a given school. This led to the discovery of “antivaxer” groups who were fighting legislation that required children to be vaccinated. Eventually, she was asked by the Obama administration to study the techniques that ISIS was using to gain recruits online because they saw the same set of techniques that the “anti vaxxers” were using. In her report for Just Security, she writes that plain old truth doesn’t stand up well against lies. She cited “a troubling statistic from a recent MIT study: on Twitter, lies are 70% more likely to be retweeted than facts.”
What’s more, a false story reaches 1,500 people six times quicker, on average, than a true story. And while false stories outperform the truth on every subject—including business, terrorism and war, science and technology, and entertainment—fake news about politics regularly does best. This is why hyper-partisan, political propaganda is a popular tactic in information warfare. The goal is not to fool people into believing any one individual lie. It’s to overwhelm individuals’ ability to determine what’s true, to create chaos,
Eventually, she began researching what the Russians did in the 2016 elections.
What she found the Russians doing was not exactly promoting Trump’s candidacy, which might have been a short-term goal, but rather they were playing a “long-game” of using simple narratives to create tribes and then mobilizing them. The Russians created pages on something that an American might take pride in, such as being from Texas or from the South. Or being Christian. Or African-American. Or LBGTQ. Or driving trucks. They’d create rather innocuous Facebook pages that seemed to align with people’s interests. Much like a publisher, they’d create content that attracted people’s interest. If content was bait, they paid close attention to the people taking the bait and hook them.
In a New York Times Op-Ed, DiResta wrote:
Russia was able to masquerade successfully as a collection of American media entities, managing fake personas and developing communities of hundreds of thousands, building influence over a period of years and using it to manipulate and exploit existing political and societal divisions.
In the podcast, Sam Harris speculates that the Russians were having great fun coming up with stories, much like a writer’s room in Hollywood coming up with jokes for a sit-com. Instead of jokes, they were coming up with exploits that would seem truer than true to Americans with certain beliefs. It makes me think of “The National Lampoon” or “Mad Magazine” and how fun it must be to write parodies that exploit what others will take too seriously.
It’s a perverse kind of genius to see this other side of America so clearly. You wouldn’t expect an outsider to be so effective and fool so many by infiltrating social media platforms that Americans think belong to them. The Russians didn’t care about the usual subjects for political activism such as electing candidates or identifying with one side of an issue. They didn’t care about what for most people is the content of politics. Instead they used content to make Americans angry and turn on each other. And somehow that made people feel good about themselves, that things were right.
What it proves to me is that the Russians understand people much better than those who run Facebook and Twitter. The Russians see more deeply in darkness where Americans see only in light. The Russians know Americans the way predators know prey. Hunters, like comics and trolls, know what makes the target vulnerable, and the prey is unaware that they are being hunted. That is also how the confidence man achieves what we wants without anyone even knowing.
Exploiting the endless varieties of vulnerability is the subject of Herman Melville’s “The Confidence Man: His Masquerade” (1857), a story of a man who wears many masks, many disguises among all variety of people who are passengers on the journey of a Mississippi river boat, named Fidèle (faith in Latin). A steamboat was filled with a swarm of people, just like on social media:
As among Chaucer’s Canterbury pilgrims, or those oriental ones crossing the Red Sea towards Mecca in the festival month, there was no lack of variety. Natives of all sorts, and foreigners; men of business and men of pleasure; parlor men and backwoodsmen; farm-hunters and fame-hunters; heiress-hunters, gold-hunters, buffalo-hunters, bee-hunters, happiness-hunters, truth-hunters, and still keener hunters after all these hunters. Fine ladies in slippers, and moccasined squaws; Northern speculators and Eastern philosophers; English, Irish, German, Scotch, Danes; Santa Fé traders in striped blankets, and Broadway bucks in cravats of cloth of gold; fine-looking Kentucky boatmen, and Japanese-looking Mississippi cotton-planters; Quakers in full drab, and United States soldiers in full regimentals; slaves, black, mulatto, quadroon; modish young Spanish Creoles, and old-fashioned French Jews; Mormons and Papists Dives and Lazarus; jesters and mourners, teetotalers and convivialists, deacons and blacklegs; hard-shell Baptists and clay-eaters; grinning negroes, and Sioux chiefs solemn as high-priests. In short, a piebald parliament, an Anacharsis Cloots congress of all kinds of that multiform pilgrim species, man.
And they are all vulnerable as prey to the confidence man who first appears in a cream-colored suit as a mute stranger who writes on his slate such innocuous sayings as “charity thinketh no evil” and “charity believeth in all things.” People wonder who he is and what he is doing but he is also just ignored. Later, this stranger approaches a well-dressed woman who is reading the New Testament. He gets her attention and confides in her that “it is very solitary for a brother here.” He continues:
“I prefer the company, however silent, of a brother or sister in good standing. By the way, madam, may I ask if you have confidence?”
“Really, sir—why, sir—really—I—”
“Could you put confidence in me for instance?”
She asks how she could possibly put confidence in an entire stranger, to which he takes offense that she should see him as such a stranger. She says that she’d like to befriend him but he answers that he cannot be friends with someone who does not have confidence in him. She admits begrudgingly that she has “some” confidence in him.
“Nay, nay, you have none—none at all. Pardon, I see it. No confidence. Fool, fond fool that I am to seek it!”
“You are unjust, sir,” rejoins the good lady with heightened interest; “but it may be that something untoward in your experiences has unduly biased you. Not that I would cast reflections. Believe me, I—yes, yes—I may say—that—that——”
“That you have confidence? Prove it. Let me have twenty dollars.”
“Twenty dollars!”
“There, I told you, madam, you had no confidence.”
She eventually asks what he wants twenty dollars for and he tells her that he wants it “for the widow and the fatherless. I am traveling agent of the Widow and Orphan Asylum, recently founded among the Seminoles.” She not only gives it to him but regrets she cannot give him more. She thinks that her faith in the goodness of other people was affirmed while he has proved her faith was foolish.
So how do we protect people from what the Russians can do, from what fracker-activists can do on social media, from what we do to ourselves? I don’t think government regulations or artificial intelligence will keep social media platforms or us from being fooled again and again.
We think we see ourselves as others see us, clearly and in good light. The phrase “to see through a glass darkly” comes to mind — the distortion in a mirror should remind us that how we see ourselves is imperfect, just as how we are seen by others. The quote comes from Paul in I Corinthians : “For now we see through a glass, darkly; but then face to face: now I know in part; but then shall I know even as also I am known.” Social media is more or less a distorting mirror, an attraction found at a funhouse or carnival, and we should know not to trust what we see reflected there.
Photo credit: Gaius Cornelius [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0) or GFDL (http://www.gnu.org/copyleft/fdl.html)%5D, from Wikimedia Commons
Dale Dougherty 