1 An Optimistic SkepticRead more at location 40
Note: bill flack è un superforecaster. come lui ce ne sono altri. domande tipo a cui risponde un s.: la russia annetterà i territori ucraini entro tre anni? l india entrerà nel consiglio di sicurezza onu tra un anno? quali paesi abbandoneranno l euro nei prossimi 5 anni? scopo del libro: spiegare il modus operandi dei s. e xchè sono tali. xchè bill nn è un editorialista del nyt? nn lo sappiamo xchè nn abbiamo un track record degli editorialisti solo opinioni su opinioni ma vaghe nn testabili. tipo: se la nato aprirà all india la russia reagirà esponendoci ad una nuova guerra fredda. ci interessa di più sapere se la juve ha fatto un buon acquisto o sapere se ci sarà un genocidio in sud sudan? sembrerebbe che la juve ci stia molto più a cuore. s. ha delle qualità che possono essere coltivate da tutti. l esperto prevede come una scimmia coi dadi? la ventennale ricerca di tetlock sfiora qs paradosso ma dice anche qlcs di più costruttivo.. l esperto fa un pochino meglio del profano solo l orizzonte 3/5 anni lo fa diventare scimmia gli eclettici fanno meglio degli specialistici. dobbiamo fidarci dell esperto? coltiviamo uno scettico ottimismo. xchè scetticismo? prendi la primavera araba nasce con una storia come tante. avrebbe potuta succedere l anno prima. facile da razionalizzare ma difficile da prevedere. quante cose può fars un battito d ali di farfalla! teoria del caos: nella simulazione pc di sistemi complessi basta variare di poco un dato e gli esiti s invertono. laplace: più sappiamo più sapremo prevedere. se sappiamo tutto del presente sapremo tutto del futuro: il mondo è un orologio. meccanicismo lorentz: no il mondo è una nuvola. i feedback radicali ci espongono a variazioni infinitesimali. non sapremo mai come evolve una nuvola. paradosso: oggi gli scienziati sanno di più ma sono più scettici sulle previsioni... complessità: legge goodheart/lucas: la realtà cambia nel momento stesso in cui viene prevista. x i siatemi instabili la previsione è impossibile. x altri viene a dipendere dalle farfalle ed è quindi difficile. altri sistemi sono più stabili. xchè ottimismo? xchè ci sono cose che si possono prevedere. x es. se ci sará traffico x andare al mare... le assicurazioni fanno molti soldi prevedendo con successo... il mondo è un misto di orologi e nuvole tutto è prob e margine d errore.. track record. essenziali x migliorare ma anche poco adottati. forse prevale lo scettico/scettico. pochi rivelano l accuratezza e quasi nessuno giudica l esperto in base a quella.. il problema della domanda: pochi chiedono conto delle evidenze... poca misurazione poca revisione pochi miglioramenti. distinguiamo: previsione x migliorare la conoscenza. previsione x divertirsi. previsione fatta x autoavverarsi. solo le prome due richiedono misurazione. ci sono anche p. fatte x impressionare: il superconsulente o il supereditorialista. anche qui il r.t. è inutile. altre confortano il militante come un bagno caldo. niente rt ovviamente. ottimismo: conoscendo tante distorsioni sappiamo dove migliorare. torneo: 5 squadre che x 4 anni rispondono a una domanda al giorno su affari e politica con una p. il metodo è libero. il gruppo deu superf. batte tutti il gruppo di controllo come il gruppo accademici. conclusione 1: la capacitá previsionale esiste ed è misurabile e nn si identifica con la potenza accademica... coclusione 2: la capacità p. nn è un talento ma un modo di pensare. un modo di aggiornare le proprie credenze. conoscere la differenza tra 60/40 e 40/60. l esperto conosce i nessi ma il superf sa pesarli. xchè progressi tanto lenti? psicologia: crediamo di conoscere quel che nn conosciamo (x esempio se panebianco è un buon p.). la differenza nn la fa l ideologia. nn la fa nemmeno l accesso ai dati e nemmeno l intelligenza. la matematica nn viene mai usata. il s. nn è uno sgobbone. conta COME si pensa. il s. è autocritico. sa correggersi. vuole migliorarsi. qs significa che è curioso e aprrto. classifica 1 s. 2 algoritmo statistico 3 esperto 4 persona comune. ma l algoritmo afgidabile è raramente disponibile x il problema da affrontare al momento. ia: nel 65 sembrava prossima. oggi siamo più cauti. il s. del futuro: un uomo col pc. l uomo fa sintesi della complessità che manda in palla un pc. il pc corregge i bias dell uomo. Edit
When we think about changing jobs, getting married, buying a home, making an investment, launching a product, or retiring, we decide based on how we expect the future will unfold. These expectations are forecasts.Read more at location 42
If you are a White House staffer, you might find him in the Oval Office with the president of the United States, talking about the Middle East. If you are a Fortune 500 CEO, you might spot him in Davos, chatting in the lounge with hedge fund billionaires and Saudi princes. And if you don’t frequent the White House or swanky Swiss hotels, you can read his New York Times columns and bestselling books that tell you what’s happening now, why, and what will come next.1 Millions do.Read more at location 44
Like Tom Friedman, Bill Flack forecasts global events. But there is a lot less demand for his insights.Read more at location 49
he went to the University of Arizona. He was aiming for a PhD in math, but he realized it was beyond his abilitiesRead more at location 53
then got a job with the Department of Agriculture and stayed for a while.Read more at location 56
Bill has answered roughly three hundred questions like “Will Russia officially annex additional Ukrainian territory in the next three months?” and “In the next year, will any country withdraw from the eurozone?”Read more at location 58
“Will North Korea detonate a nuclear device before the end of this year?” “How many additional countries will report cases of the Ebola virus in the next eight months?” “Will India or Brazil become a permanent member of the UN Security Council in the next two years?”Read more at location 61
“Will NATO invite new countries to join the Membership Action Plan (MAP) in the next nine months?” “Will the Kurdistan Regional Government hold a referendum on national independence this year?” “If a non-Chinese telecommunications firm wins a contract to provide Internet services in the Shanghai Free Trade Zone in the next two years, will Chinese citizens have access to Facebook and/or Twitter?”Read more at location 63
But he does his homework. He gathers facts, balances clashing arguments, and settles on an answer.Read more at location 67
No one bases decisions on Bill Flack’s forecasts, or asks Bill to share his thoughts on CNN.Read more at location 68
that’s unfortunate. Because Bill Flack is a remarkable forecaster.Read more at location 69
each one of Bill’s predictions has been dated, recorded, and assessed for accuracy by independent scientific observers.Read more at location 70
engineers and lawyers, artists and scientists, Wall Streeters and Main Streeters, professors and students.Read more at location 72
accuracy of Friedman’s forecasting has never been rigorously tested.Read more at location 76
But there are no hard facts about Tom Friedman’s track record, just endless opinions—and opinions on opinions.Read more at location 79
make critical decisions on the basis of forecasts whose quality is unknown.Read more at location 83
the average expert was roughly as accurate as a dart-throwing chimpanzee. You’ve probably heard that one before.Read more at location 90
It goes like this: A researcher gathered a big group of experts—academics, pundits, and the like—to make thousands of predictions about the economy, stocks, elections, wars, and other issues of the day. Time passed, and when the researcher checked the accuracy of the predictions, he found that the average expert did about as well as random guessing. Except that’s not the punch line because “random guessing” isn’t funny. The punch line is about a dart-throwing chimpanzee. Because chimpanzees are funny. I am that researcherRead more at location 92
My study was the most comprehensive assessment of expert judgmentRead more at location 97
Open any newspaper, watch any TV news show, and you find experts who forecastRead more at location 101
With few exceptions, they are not in front of the cameras because they possess any proven skill at forecasting.Read more at location 102
The one undeniable talent that talking heads have is their skill at telling a compelling story with conviction, and that is enough. Many have become wealthy peddling forecasting of untested value to corporate executives, government officials, and ordinary people who would never think of swallowing medicine of unknown efficacy and safety but who routinely pay for forecasts that are as dubious as elixirs sold from the back of a wagon.Read more at location 104
What my research had shown was that the average expert had done little better thanRead more at location 109
approaching the dart-throwing-chimpanzee level three to five years out.Read more at location 111
The message became “all expert forecasts are useless,” which is nonsense.Read more at location 115
and that any intelligent, open-minded, and hardworking person can cultivate the requisite skills.Read more at location 123
To understand the “skeptic” half of that label, consider a young Tunisian man pushing a wooden handcart loaded with fruits and vegetables down a dusty road to a market in the Tunisian town of Sidi Bouzid. When the man was three, his father died. He supports his family by borrowing money to fill his cart, hoping to earn enough selling the produce to pay off the debt and have a little left over. It’s the same grind every day. But this morning, the police approach the man and say they’re going to take his scales because he has violated some regulation. He knows it’s a lie. They’re shaking him down. But he has no money. A policewoman slaps him and insults his dead father. They take his scales and his cart. The man goes to a town office to complain. He is told the official is busy in a meeting. Humiliated, furious, powerless, the man leaves. He returns with fuel. Outside the town office he douses himself, lights a match, and burns. Only the conclusion of this story is unusual. There are countless poor street vendors in Tunisia and across the Arab world.Read more at location 125
But this particular humiliation, on December 17, 2010, caused Mohamed Bouazizi, aged twenty-six, to set himself on fire, and Bouazizi’s self-immolation sparked protests. The police responded with typical brutality. The protests spread. Hoping to assuage the public, the dictator of Tunisia, President Zine el-Abidine Ben Ali, visited Bouazizi in the hospital. Bouazizi died on January 4, 2011. The unrest grew. On January 14, Ben Ali fled to a cushy exile in Saudi Arabia, ending his twenty-three-year kleptocracy. The Arab world watched, stunned. Then protests erupted in Egypt, Libya, Syria, Jordan, Kuwait, and Bahrain. After three decades in power, the Egyptian dictator Hosni Mubarak was driven from office. Elsewhere, protests swelled into rebellions, rebellions into civil wars.Read more at location 134
having made his name in journalism as a New York Times correspondent in Lebanon.Read more at location 144
Maybe, given how much Friedman knew about the region, he would have mused that poverty and unemployment were high,Read more at location 146
But an observer could have drawn exactly the same conclusion the year before.Read more at location 148
Does the Flap of a Butterfly’s Wings in Brazil Set Off a Tornado in Texas?”Read more at location 151
Lorenz had discovered by accident that tiny data entry variations in computer simulations of weather patterns—like replacing 0.506127 with 0.506—could produce dramatically different long-term forecasts. It was an insight that would inspire “chaos theory”: in nonlinear systems like the atmosphere, even small changes in initial conditions can mushroom to enormous proportions.Read more at location 152
He meant that if that particular butterfly hadn’t flapped its wingsRead more at location 157
For centuries, scientists had supposed that growing knowledge must lead to greater predictabilityRead more at location 162
We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes.Read more at location 166
If it knew everything about the present, Laplace thought, it could predict everything about the future.Read more at location 171
High school science tells us that clouds form when water vapor coalesces around dust particles. This sounds simple but exactly how a particular cloud develops—the shape it takes—depends on complex feedback interactions among droplets. To capture these interactions, computer modelers need equations that are highly sensitive to tiny butterfly-effect errors in data collection.Read more at location 173
This is a big reason for the “skeptic” half of my “optimistic skeptic”Read more at location 179
A woman living in a Kansas City suburb may think Tunisia is another planet, and her life has no connection to it, but if she were married to an air force navigator who flies out of the nearby Whiteman Air Force Base, she might be surprised to learn that one obscure Tunisian’s actions led to protests, that led to riots, that led to the toppling of a dictator, that led to protests in Libya, that led to a civil war, that led to the 2012 NATO intervention, that led to her husband dodging antiaircraft fire over Tripoli.Read more at location 181
Why did she leave home at 6:30? She didn’t want to get stuck in rush hour.Read more at location 197
She expected the people who said they would join the 10:30 conference call to do so, and she was right.Read more at location 199
When the woman went to Amazon, the website highlighted certain products it thought she would like,Read more at location 205
Google personalizes search results by putting what it thinks you will find most interestingRead more at location 207
Kansas City Life Insurance Company is in the business of forecasting disability and death, and it does a good job. That doesn’t mean it knows precisely when I will die, but it does have a good idea of how long someone of my age and profile—sex, income, lifestyle—is likely to live. Kansas City Life was founded in 1895. If its actuaries weren’t good forecasters, it would have gone bankrupt long ago.Read more at location 208
I just Googled tomorrow’s sunrise and sunset times for Kansas City, Missouri, and got them down to the minute.Read more at location 212
A good restaurant is very likely to open its doors when it says it will, but it may not, for any number of reasons,Read more at location 215
Weather forecasts are typically quite reliable, under most conditions, looking a few days ahead, but they become increasingly less accurateRead more at location 225
further we try to look into the future, the harder it is to see.Read more at location 228
separating the predictable from the unpredictable is difficult work.Read more at location 233
meteorologists are able to sharpen their understanding of how weather works and tweak their models. Then they try again. Forecast, measure, revise. Repeat. It’s a never-ending process of incremental improvement that explains why weather forecasts are good and slowly getting better.Read more at location 235
Big leaps in computing power and continued refinement of forecasting models may nudge the limits a little furtherRead more at location 239
Accuracy is seldom determined after the fact and is almost never done with sufficient regularityRead more at location 246
“I have been struck by how important measurement is to improving the human condition,” Bill Gates wrote. “You can achieve incredible progress if you set a clear goal and find a measure that will drive progress toward that goal. … This may seem basic, but it is amazing how often it is not done and how hard it is to get right.”Read more at location 252
You might think the goal of forecasting is to foresee the futureRead more at location 257
Think of CNBC’s Jim Cramer with his “booyah!” shtick, or John McLaughlin, the host of The McLaughlin Group, bellowing at his panelists to predict the likelihood of an event “on a scale from zero to ten, with zero representing zero possibility and ten representing complete metaphysical certitude!”Read more at location 258
forecasts are used to advance political agendas and galvanize action—Read more at location 260
There is also dress-to-impress forecasting—which is what banks deliver when they pay a famous punditRead more at location 262
It’s a messy situation, which doesn’t seem to be getting better. And yet this stagnation is a big reason why I am an optimisticRead more at location 265
For scientists, not knowing is exciting. It’s an opportunity to discover;Read more at location 268
all we have to do is set a clear goal—accuracy!—and get serious about measuring.Read more at location 270
I’ve been doing that for much of my career. The research that produced the dart-throwing-chimpanzee result was phase one. Phase two started in the summer of 2011, when my research (and life) partner Barbara Mellers and I launched the Good Judgment Project and invited volunteers to sign up and forecast the future. Bill Flack responded. So did a couple of thousand others that first year, and thousands more in the four years that followed. Cumulatively, more than twenty thousand intellectually curious laypeople tried to figure out if protests in Russia would spread, the price of gold would plummet, the Nikkei would close above 9,500, war would erupt on the Korean peninsula, and many other questions about complex, challenging global issues. By varying the experimental conditions, we could gauge which factors improved foresight, by how much, over which time frames, and how good forecasts could become if best practices were layered on each other.Read more at location 271
part of a much larger research effort sponsored by the Intelligence Advanced Research Projects Activity (IARPA).Read more at location 279
intelligence community that reports to the director of National IntelligenceRead more at location 281
By one rough estimate, the United States has twenty thousand intelligence analysts assessing everything from minute puzzles to major events such as the likelihood of an Israeli sneak attack on Iranian nuclear facilities or the departure of Greece from the eurozone.Read more at location 283
intelligence community, like so many major producers of forecasting, has never been keen on spending money to figure that out.Read more at location 286
To change that, IARPA created a forecasting tournament in which five scientific teams led by top researchers in the field would compete to generate accurate forecasts on the sorts of tough questions intelligence analysts deal with every day. The Good Judgment Project was one of those five teams. Each team would effectively be its own research project, free to improvise whatever methods it thought would work, but required to submit forecasts at 9 a.m. eastern standard time every day from September 2011 to June 2015. By requiring teams to forecast the same questions at the same time, the tournament created a level playing field—and a rich trove of data about what works, how well, and when. Over four years, IARPA posed nearly five hundred questions about world affairs.Read more at location 289
In year 1, GJP beat the official control group by 60%. In year 2, we beat the control group by 78%. GJP also beat its university-affiliated competitors, including the University of Michigan and MIT, by hefty margins, from 30% to 70%, and even outperformed professional intelligence analysts with access to classified data. After two years, GJP was doing so much better than its academic competitors that IARPA dropped the other teams.Read more at location 296
One, foresight is real. Some people—people like Bill Flack—have it in spades.Read more at location 300
One result that particularly surprised me was the effect of a tutorial coveringRead more at location 305
“it’s the difference between a consistent winner who’s making a living, or the guy who’s going broke all the time.”Read more at location 310
The difference between heavyweights and amateurs, she said, is that the heavyweights know the difference between a 60⁄40 bet and a 40⁄60 bet.Read more at location 312
When physicians finally accepted that their experience and perceptions were not reliable means of determining whether a treatment works, they turned to scientific testing—and medicine finally started to make rapid advances.Read more at location 317
In the late 1980s I worked out a methodology and conducted what was, at the time, the biggest test of expertRead more at location 320
Why are they so good? That question runs through chapters 5 through 9.Read more at location 327
many have advanced degrees in mathematics and science. So is the secret arcane math? No.Read more at location 329
so you might be tempted to attribute their success to spending endless hours on the job. Yet that too would be a mistake.Read more at location 331
Superforecasting does require minimum levels of intelligence, numeracy, and knowledge of the world, but anyone who reads serious books about psychological research probably has those prerequisites.Read more at location 332
superforecasting demands thinking that is open-minded, careful, curious, and—above all—self-critical. It also demands focus.Read more at location 335
maybe you think this is all hopelessly outdated. After all, we live in an era of dazzlingly powerful computers,Read more at location 341
In 1954, a brilliant psychologist, Paul Meehl wrote a small book that caused a big stir.12 It reviewed twenty studies showing that well-informed experts predicting outcomes—whether a student would succeed in college or a parolee would be sent back to prison—were not as accurate as simple algorithmsRead more at location 344
The point is now indisputable: when you have a well-validated statistical algorithm, use it.Read more at location 349
But spectacular advances in information technology suggest we are approaching a historical discontinuityRead more at location 352
In 1997 IBM’s Deep Blue beat chess champion Garry Kasparov. Now, commercially available chess programs can beat any human. In 2011 IBM’s Watson beat Jeopardy! champions Ken Jennings and Brad Rutter. That was a vastly tougher computing challenge, but Watson’s engineers did it.Read more at location 353
In 1965 the polymath Herbert Simon thought we were only twenty years away from a world in which machines could do “any work a man can do,”Read more at location 360
Ferrucci—who has worked in artificial intelligence for thirty years—is more cautious today.Read more at location 362
Even with computers making galloping advances, the sort of forecasting that superforecasters do is a long way off.Read more at location 370
“there’s a difference between mimicking and reflecting meaning and originating meaning,”Read more at location 373
we will also see more and more syntheses, like “freestyle chess,Read more at location 376
The result is a combination that can (sometimes) beat both humans and machines.Read more at location 377
that makes so many policy debates so puerile: “I’ll counter your Paul Krugman polemic with my Niall Ferguson counterpolemic, and rebut your Tom Friedman op-ed with my Bret Stephens blog.”Read more at location 379
“I think it’s going to get stranger and stranger” for people to listen to the advice of experts whose views are informed only by their subjective judgment.Read more at location 381
“So what I want is that human expert paired with a computer to overcome the human cognitive limitations and biases.”Read more at location 383
need to blend computer-based forecasting and subjective judgmentRead more at location 385
