Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
You have 45 highlighted passages
You have 48 notes
Last annotated on October 13, 2016
The math-powered applications powering the data economy were based on choices made by fallible human beings. Some of these choices were no doubt made with the best intentions. Nevertheless, many of these models encoded human prejudice, misunderstanding, and bias into the software systems that increasingly managed our lives. Like gods, these mathematical models were opaque, their workings invisible to all but the highest priests in their domain: mathematicians and computer scientists. Their verdicts, even when wrong or harmful, were beyond dispute or appeal. And they tended to punish the poor and the oppressed in our society, while making the rich richer.Read more at location 95
I post a petition on my Facebook page. Which of my friends will see it on their news feed? I have no idea. As soon as I hit send, that petition belongs to Facebook, and the social network’s algorithm makes a judgment about how to best use it. It calculates the odds that it will appeal to each of my friends.Read more at location 2644
While Facebook may feel like a modern town square, the company determines, according to its own interests, what we see and learn on its social network.Read more at location 2650
By tweaking its algorithm and molding the news we see, can Facebook game the political system?Read more at location 2655
Facebook conducted experiments to hone a tool they called the “voter megaphone.” The idea was to encourage people to spread word that they had voted. This seemed reasonable enough. By sprinkling people’s news feeds with “I voted” updates, Facebook was encouraging Americans—more than sixty-one million of them—to carry out their civic duty and make their voices heard.Read more at location 2656
Facebook researchers were studying how different types of updates influenced people’s voting behavior. No researcher had ever worked in a human laboratory of this scale. Within hours, Facebook could harvest information from tens of millions of people, or more, measuring the impact that their words and shared links had on each other. And it could use that knowledge to influence people’s actions, which in this case happened to be voting.Read more at location 2662
Google, Apple, Microsoft, Amazon, and cell phone providers like Verizon and AT&T,Read more at location 2666
The government regulates them, or chooses not to, approves or blocks their mergers and acquisitions, and sets their tax policies (often turning a blind eye to the billions parked in offshore tax havens). This is why tech companies, like the rest of corporate America, inundate Washington with lobbyists and quietly pour hundreds of millions of dollars in contributions into the political system.Read more at location 2669
The Facebook campaign started out with a constructive and seemingly innocent goal: to encourage people to vote. And it succeeded. After comparing voting records, researchers estimated that their campaign had increased turnout by 340,000 people. That’s a big enough crowd to swing entire states, and even national elections. George W. Bush, after all, won in 2000 by a margin of 537 votes in Florida.Read more at location 2673
Facebook’s potency comes not only from its reach but also from its ability to use its own customers to influence their friends.Read more at location 2677
Would people encourage their friends to vote, and would this affect their behavior? According to the researchers’ calculations, seeing that friends were participating made all the difference.Read more at location 2683
About 20 percent of the people who saw that their friends had voted also clicked on the “I Voted” button.Read more at location 2685
You might argue that newspapers have exerted similar power for eons. Editors pick the front-page news and decide how to characterize it. They choose whether to feature bombed Palestinians or mourning Israelis, a policeman rescuing a baby or battering a protester. These choices can no doubt influence both public opinion and elections. The same goes for television news. But when the New York Times or CNN covers a story, everyone sees it. Their editorial decision is clear, on the record. It is not opaque. And people later debate (often on Facebook) whether that decision was the right one. Facebook is more like the Wizard of Oz: we do not see the human beings involved.Read more at location 2696
The potential for Facebook to hold sway over our politics extends beyond its placement of news and its Get Out the Vote campaigns. In 2012, researchers experimented on 680,000 Facebook users to see if the updates in their news feeds could affect their mood. It was already clear from laboratory experiments that moods are contagious.Read more at location 2705
Using linguistic software, Facebook sorted positive (stoked!) and negative (bummed!) updates. They then reduced the volume of downbeat postings in half of the news feeds, while reducing the cheerful quotient in the others.Read more at location 2708
“Emotional states can be transferred to others…, leading people to experience the same emotions without their awareness.”Read more at location 2712
What would occur if they played with people’s emotions on Election Day?Read more at location 2714
Two researchers, Robert Epstein and Ronald E. Robertson, recently asked undecided voters in both the United States and India to use a search engine to learn about upcoming elections. The engines they used were programmed to skew the search results, favoring one party over another.Read more at location 2720
Modern consumer marketing, however, provides politicians with new pathways to specific voters so that they can tell them what they know they want to hear. Once they do, those voters are likely to accept the information at face value because it confirms their previous beliefs, a phenomenon psychologists call confirmation bias.Read more at location 2756
In The Selling of the President, which followed Richard Nixon’s 1968 campaign, the journalist Joe McGinniss introduced readers to the political operatives working to market the presidential candidate like a consumer good.Read more at location 2762
as time went on, politicians wanted a more detailed approach, one that would ideally reach each voter with a personalized come-on. This desire gave birth to direct-mail campaigns. Borrowing tactics from the credit card industry, political operatives built up huge databases of customers—voters, in this case—and placed them into various subgroups, reflecting their values and their demographics.Read more at location 2765
The convergence of Big Data and consumer marketing now provides politicians with far more powerful tools.Read more at location 2769
In July of 2011, more than a year before President Obama would run for reelection, a data scientist named Rayid Ghani posted an update on LinkedIn: Hiring analytics experts who want to make a difference. The Obama re-election campaign is growing the analytics team to work on high-impact large-scale data mining problems. We have several positions available at all levels of experience. Looking for experts in statistics, machine learning, data mining, text analytics, and predictive analytics to work with large amounts of data and help guide election strategy.Read more at location 2775
Ghani’s science translated perfectly into politics. Those fickle shoppers who switched brands to save a few cents, for example, behaved very much like swing voters. In the supermarket, it was possible to estimate how much it would cost to turn each shopper from one brand of ketchup or coffee to another more profitable brand. The supermarket could then pick out, say, the 15 percent most likely to switch and provide them with coupons. Smart targeting was essential. They certainly didn’t want to give coupons to shoppers who were ready to pay full price. That was like burning money.Read more at location 2794
Four years later, Hillary Clinton’s campaign built upon the methodology established by Obama’s team. It contracted a microtargeting start-up, the Groundwork, financed by Google chairman Eric Schmidt and run by Michael Slaby, the chief technology officer of Obama’s 2012 campaign. The goal, according to a report in Quartz, was to build a data system that would create a political version of systems that companies like Salesforce.com develop to manage their millions of customers. The appetite for fresh and relevant data, as you might imagine, is intense.Read more at location 2820
information to develop psychographic analyses of more than forty million voters, ranking each on the scale of the “big five” personality traits: openness, conscientiousness, extroversion, agreeableness, and neuroticism.Read more at location 2827
Similar calculations, on a macro scale, have been going on for decades, as campaigns plot their TV spending. As polling num bers change, they might cut ads in Pittsburgh and move those dollars to Tampa or Las Vegas. But with microtargeting, the focus shifts from the region to the individual.Read more at location 2845
The campaigns use similar analysis to identify potential donors and to optimize each one.Read more at location 2848
These tactics aren’t limited to campaigns. They infect our civic life, with lobbyists and interest groups now using these targeting methods to carry out their dirty work.Read more at location 2857
In 2015, the Center for Medical Progress, an antiabortion group, posted videos featuring what they claimed was an aborted fetus at a Planned Parenthood clinic. The videos asserted that Planned Parenthood doctors were selling baby parts for research, and they spurred a wave of protest, and a Republican push to eliminate the organization’s funding.Read more at location 2858
with microtargeting, antiabortion activists could continue to build an audience for the video, despite the flawed premise, and use it to raise funds to fight Planned Parenthood.Read more at location 2864
According to Zeynep Tufekci, a techno-sociologist and professor at the University of North Carolina, these groups pinpoint vulnerable voters and then target them with fear-mongering campaigns, scaring them about their children’s safety or the rise of illegal immigration.Read more at location 2868
Successful microtargeting, in part, explains why in 2015 more than 43 percent of Republicans, according to a survey, still believed the lie that President Obama is a Muslim.Read more at location 2870
Television delivers the broader, and accountable, messaging, while microtargeting does its work in the shadows. But even television is moving toward personalized advertising. New advertising companies like Simulmedia, in New York, assemble TV viewers into behavioral buckets, so that advertisers can target audiences of like-minded people, whether hunters, pacifists, or buyers of tank-sized SUVs.Read more at location 2876
it will become harder to access the political messages our neighbors are seeing—and as a result, to understand why they believe what they do, often passionately.Read more at location 2880
The result of these subterranean campaigns is a dangerous imbalance. The political marketers maintain deep dossiers on us, feed us a trickle of information, and measure how we respond to it. But we’re kept in the dark about what our neighbors are being fed.Read more at location 2887
This asymmetry of information prevents the various parties from joining forces—which is precisely the point of a democratic government.Read more at location 2890
As I write this, the entire voting population that matters lives in a handful of counties in Florida, Ohio, Nevada, and a few other swing states.Read more at location 2895
The programs have already predicted our voting behavior, and any attempt to change it is not worth the investment.Read more at location 2901
People who are expected to be voters but who, for one reason or another, skip an election find themselves lavished with attention the next time round.Read more at location 2906
As is often the case with WMDs, the very same models that inflict damage could be used to humanity’s benefit. Instead of targeting people in order to manipulate them, it could line them up for help. In a mayoral race, for example, a microtargeting campaign might tag certain voters for angry messages about unaffordable rents. But if the candidate knows these voters are angry about rent, how about using the same technology to identify the ones who will most benefit from affordable housing and then help them find it?Read more at location 2909
