Dell’economista viene detto in modo un po’ canzonatorio che è uno “scienziato senza laboratorio”.
L’accusa appare fondata a Jim Manzi il quale - nel suo lavoro “Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society” – la estende a tutte le “scienze sociali non sperimentali”.
I fisici sono noiosi, difficile discutere, hanno sempre ragione loro. Gli storici sono molto più interessanti, uscire con loro a cena genera discussioni infinite, abbiamo sempre mille obiezioni a quanto ci dicono, non riescono mai a convincerci per quanto siano eruditi. E con gli economisti come siamo messi?
Se non fanno gli gnorri trincerandosi dietro un linguaggio fintamente matematico, anche loro possono essere una buona compagnia.
Il “caso del presidente” esprime al meglio il concetto…
… Imagine that a hypothetical US president is considering his options vis-à-vis Iran’s rapidly developing nuclear weapons program. First a science adviser enters the room and predicts that if the Iranians take a certain quantity of fissile material and compress it into a sphere of a particular size under specific conditions, then it will cause an explosion large enough to destroy a major city. Next a historian enters the room and predicts that if external attempts are made to thwart Iranian nuclear ambitions, then a popular uprising will ensue sooner or later, and force changes in governments until Iran has achieved nuclear capability. The president would be incredibly irresponsible to begin debating nuclear physics with his science adviser, even if the president happened to have trained as a physicist. Conversely, the president would be incredibly irresponsible not to begin a debate with the historian. This likely would include having several historians present different perspectives… Next an economist walks into the room… Superficially she might sound a lot more like the physicist. She would use lots of empirical data, equations, and technical language… But lots of things would arguably remain outside the grasp of formal models…
Come giustifica le sue tesi l’economista?
… (1) a priori beliefs about human nature, and conclusions that are believed to be logically derivable from them, (2) analysis of historical data, which is to say, data-driven theory-building, and (3) a review of the track record of prior predictions made using the predictive rule in question…
Al fisico basta ed avanza il punto tre. Basta che dica:
… Please view the following film taken from a long series of huge explosions that result when independent evaluators combine the materials I described in the manner I described… The reason the physicist need concentrate only on controlled experiments is that these are accepted as the scientific gold standard for testing theories
Gli esperimenti controllati per lui sono facili e spesso l’esito è univoco. Questa fortuna non bacia l’economista:
… so many things change in a macroeconomic event that it is not realistic to isolate the causal impact of any one factor…
A volte il povero economista sembra travestire la retorica in abiti analitici. Nemmeno le verità più consolidate della disciplina raccolgono l’unanimità dei consensi…
… Mankiw summarized fourteen findings that have achieved widespread acceptance among economists. Among them are: Fiscal policy (e.g., tax cut and/or government expenditure increase) has a significant stimulative impact on a less than fully employed economy. A large federal budget deficit has an adverse effect on the economy. A minimum wage increases unemployment among young and unskilled workers. In fact, 10 to 20 percent of practicing economists disagree with each of these assertions; but more fundamentally, even if we assume them to be correct, they are too vague to really help settle policy arguments…
L’economia è la classica scienza non sperimentale.
… although experiments can help and should be aggressively pursued, our scientific knowledge of any human social organization will remain extremely limited even when these experiments are deployed extensively…
L’economista è uno scienziato frustrato e proprio perché il suo laboratorio ha mille falle deve fare di tutto per “spremere il sangue dalle rape” cosicché si trasforma in uno sperimentatore geniale come neanche i fisici riusciranno mai a diventare senonché tanta elefantiaca genialità quasi sempre partorisce un topolino.
L’economista ha spesso a che fare con la cosiddetta “alta densità causale”, il che significa che studia fenomeni influenzati da una marea di variabili. Questo labirinto di cause acuisce il problema dell’induzione.
Prima di credersi uno scienziato l’economista era uno storico oppure si limitava a premesse scaturite dal buon senso su cui fondava teorie con esiti magari anche contro intuitivi ma che non ambivano alla dimostrazione sperimentale…
… Prior to the creation of modern social science, we simply had history, with its tradition of recording facts and making assertions based on these facts plus narrative appeals to commonly held understandings of human motivations and experiences. This was nonscientific, in that it did not make claims for the kinds of reliable, nonobvious, and useful predictive rules that characterize science. In the terms of this book, history is informed common sense…
Poi venne l’illuminismo e fu il trionfo dell’equazione…
… numerous thinkers attempted to apply scientific methods to the study of human social behavior. The French Enlightenment, in particular, was central to the creation of the modern social science ideology. Auguste Comte and Henri de Saint-Simon were explicit in arguing that the methods of natural science provided the model for developing predictive laws for human society. Comte argued that human understanding in various fields proceeded in three stages: theological, metaphysical, and finally, positive… we would say that knowledge proceeds from mythology to philosophy to science… Comte believed that humanity had achieved “positive” (i.e., scientific) understanding in various fields in the order of their complexity: mathematics, astronomy, physics, chemistry, biology, and finally sociology… It was clear to the earliest social scientists that the natural sciences of their era (astronomy, chemistry, and physics) achieved spectacular success by discovering and stating physical laws as equations…
John Stuart Mill fu il primo economista scienziato desideroso di “verificare” i suoi ragionamenti:
… He argued that despite the inability to conduct controlled experiments in social sciences, thinkers could reason from introspection to general predictive rules… In an argument somewhat akin to Sir Karl Popper’s doctrine of falsification, Mill argued that one role of empirical observation is to “verify” a given causal rule by “by comparing…
Ma ahimé, la verifica si dimostro alquanto ostica, almeno per le tesi più interessanti e divisive. Fu la morte del sogno di Mill.
… For example, a social scientist might promulgate a predictive rule that a US president will fail to win reelection if the unemployment rate exceeds 10 percent. If, in a specific future election, a president were to win reelection with 11 percent unemployment, the social scientist might observe that there was a disturbing cause created by the fact that the nation was at war and the president was viewed as an indispensable leader… But what if there are myriad “disturbing causes,” many of which are as important as the cause of interest in determining the outcome of the situation?… The problem of how we can develop nonobvious, reliable predictive rules without controlled experiments has so far been deadly to Comte and Mill’s dream of rational social science…
Forse è meglio vedere da vicino le falle tipiche delle “scienze non sperimentali. Prenderemo due casi con caratteristiche comuni:
… First, both asserted findings are nonobvious, but plausible… Second, each of these should be a very high-quality analysis…
Il primo caso:
… The first example is a regression model presented by Princeton public policy professor Larry M. Bartels in his 2008 book, Unequal Democracy…
Si tratta di una ricerca premiata con il prestigioso Kammerer Award dell’ American Political Science Association (APSA). Motivazione:
… “the care taken in the analysis” and “the rigorous application of controls.”…
La tesi sostenuta:
… The most widely discussed finding in the book was a regression analysis, based in part on an updated version of analysis from his 2004 academic paper, “Partisan Politics and the US Income Distribution,” which reviews the changes in incomes for the rich versus the poor under Democratic versus Republican presidents from 1948 to 2005. Bartels asserts that the differences in the behavior of Republican versus Democratic presidents have been a leading cause of the rich gaining relative income versus the poor, saying these presidential differences were “the most important single influence on the changing US income distribution over the past half-century.”…
Da notare che sulla relazione esaminata da Bartels (Presidenza-Diseguaglianza nei redditi) incidono moltissime variabili, esempio:
… decisions by the Congress, Supreme Court, and Federal Reserve; changes in international economic competition; technological developments that enhance some people over others; changes in immigration rates and sources; changes in social mores and beliefs; and being at war or peace… The price of oil and the increasing participation of women in the labor force…
Siamo in un caso classico di “alta densità causale”.
Bartels riconosce il problema ma poi – misteriosamente – minimizza…
… “because these long-term trends have been so glacial, and so intertwined, it is very difficult to discern their distinct effects on the shape of the income distribution”… “Fortunately, from the standpoint of political analysis, the very fact that these social and economic trends have been gradual and fairly steady implies that their effects are unlikely to be confounded with the effects of alterations in control of the White House.”…
L’ovvia questione è: perché mai tutte queste variabili non dovrebbero interferire nella relazione in oggetto? L’ottimismo di Bartels sembra ingiustificato.
Il modello proposto:
… Based on these assertions, Bartels builds a set of regression models that attempts to explain changes in income inequality in any given year as a function of six variables… One of these variables is last year’s party of the president… then last year’s change in the price of oil, and last year’s change in female labor force participation… The fourth variable is last year’s income growth at the 95th percentile (i.e., growth “trickles down”)… Finally, he understands that this short list of four factors cannot conceivably describe all of the “great many economic and social forces”… he adds two variables to his equations to fit his curves to the historical trend of the data, rather than to explain this historical trend as a function of underlying causes. One of these trend variables is the number of years since 1948 for each year, and the final variable in the model is the square of the number of years since 1948 for each year. And that’s it…
Con le ultime variabili di “tendenza” Bartels pretende di catturare tutte le forze di lungo periodo.
Ci sono altri assunti che definire problematici è poco…
… assumption that presidential actions affect income distributions for only one year… There are plausible long-term causal mechanisms that could have almost no effect for years… obviously including appointments to the Supreme Court and Federal Reserve…Did Reagan’s effects on the change in distribution of incomes in America really end in 1989?…
Ma poi c’è quanto dicevamo prima: misurare solo due variabili lasciando che le restanti siano inglobate in un trend complessivo è un assunto forte…
… assumption that out of all the potential confounding causes for inequality, only oil prices and female labor participation should be included in the model as specific causes, and that the model has captured all of the other possible causal factors through his “linear and quadratic trend terms.”…
E infatti ipotizzando un lag di due anni l’esito già cambia…
… Using the raw Census data tables, I observed that income inequality does tend to rise under Republican presidents (lagged one year—e.g., Jimmy Carter gets credit for 1981) and fall under Democratic presidents (lagged one year). But when I did the simple test of changing the lag to two years, the entire apparent effect disappears… he cites two academic papers that he believes show his assumption is “consistent with macroeconomic evidence regarding the timing of economic responses to monetary and fiscal policy changes.” But first note that a president can affect a far broader range of policies than monetary and fiscal policy—for example, regulatory decisions, Supreme Court and Federal Reserve appointments, negotiating trade treaties, antitrust enforcement, seeking out or settling wars… And these papers don’t appear to claim a one-year rather than a two-year window for the impacts they do analyze. One paper estimates that (1) the peak impact of a tax shock on GDP should be reached by one to two years after the taxes change, and thereafter continue indefinitely; and (2) the peak impact of a spending shock should not be reached until two to four years after the spending change, and then continue indefinitely. The other paper estimates that numerous effects of monetary shocks extend for two years or more…
Non dobbiamo accusare Bartels per la leggerezza del suo modello, lui è tra ipiù ferrati in circolazione, al limite dobbiamo accusare la comunità scientifica che accetta questi standard e li premia come attendibili. In un campo come questo non si puo’ fare di meglio…
… For this kind of a social reality, such model-tuning (for example, the one-year lag versus a two-year lag; including oil prices and female labor force participation versus the myriad other potential control variables; using a linear plus quadratic trend terms versus searching for additional explicit control variables, etc.) is inevitable, because the complexity of the real world overwhelms the tool of regression analysis…
Il secondo caso riguarda il legame tra aborto e criminalità. L’autore è:
… Steven Levitt, a distinguished economics professor at the University of Chicago. Levitt was awarded the John Bates Clark Medal as the best American economist under forty…
La tesi espressa:
… Among the most widely discussed passages in Freakonomics was Levitt’s assertion that a significant fraction of US crime reduction in the 1990s can be linked to changes set in motion by Roe v. Wade in 1973. The basic asserted causal mechanism is that the increase in abortions disproportionately eliminated potential future criminals…
Le potenziali variabili che potrebbero interferire:
… several fertility control technologies—most importantly the birth control pill—plus a huge variety of social trends that plausibly affect abortion rates and/or crime emerged in the same era as legalized abortion. The argument Levitt makes in his professional publications is that we can control for these other effects. But this is extremely difficult if these other effects became evident at the times, in the places, and for the population subgroups where abortion legalization had its first effects…
Levitt punta l’attenzione su alcuni “esperimenti naturali” (dei succedanei dell’esperimento controllato):
… Freakonomics presents the results of a natural experiment: the five states that liberalized abortion laws prior to Roe (Levitt terms these “early legalizers”) experienced a crime reduction prior to the nonrepeal states… + the states with the highest abortion rates in the 1970s experienced the greatest crime drops… + they note that Australia and Canada have seen similar results…
La logica della regressione è sempre quella: neutralizzare alcune variabili nel tentativo di creare una situazione “coeteris paribus”. Ma le variabili in gioco sono troppe e basta cambiare alcuni assunti per non replicare più l’esperimento…
… other academics published alternative versions of the same analysis, using slightly different assumptions, that did not show any such effect. Levitt and Donohue, of course, quickly replied by arguing that one should use their preferred specifications…
Nel caso in oggetto ci fu un’altra obiezione…
… two Federal Reserve economists published a crucial criticism in which they showed that the software implementation of the equations presented in DL 2001 had an important error and that once this was corrected and some other technical changes were made, the asserted effect of abortion on crime was no longer evident…
Anche le banche dati utilizzate hanno un loro peso…
… using a different data set massaged differently to reflect better how people moved among various states after having abortions…
Le repliche poi sono sempre problematiche…
… Other academics then attempted to replicate the same analysis for the effect of legalization of abortion in the United Kingdom. They also discovered that depending on the exact specification of data sets and assumptions made in the regression model, the effect on crime would sometimes appear, and sometimes not…
Le parole dei ricercatori inglesi che hanno provato la replica sono sintomatiche:
… The fragility of the results in this paper serve to emphasize the difficulty researchers have in identifying causal effects of social change such as abortion legalization on crime rates some years hence, particularly given the myriad of other social changes occurring over the same time and which may dilute any effect…
Una possibile conclusione:
… Once again, regression analysis cannot tell us the effects of abortion on crime, because different reasonable assumptions for the analysis lead to completely different answers…
L’unico modo di risolvere la questione sarebbe quella di fare un esperimento ma la cosa è impossibile in queste materie…
… One way to get around all of this confusion would be to run an experiment. A purposeful experiment to force a random sample of states to implement abortion legalization has never happened in American history, and almost certainly never will…
Purtroppo, gli esperimenti naturali non sono mai dei buoni sostituti degli esperimenti controllati, e anche questo caso lo conferma, basta guardare al crimine negli stati che hanno anticipato l’aborto, non sembra affatto di scorgere un andamento omogeneo…
… New York declines 35 percent, while Alaska increases 50 percent; California is down 14 percent, and Hawaii is up 11 percent; Washington is almost exactly flat. The total rate across the early legalizers goes down versus the rest of the country only because New York and California are so much larger than the other three states. The natural experiment cannot resolve the question of the causal impact of abortion on crime, either…
L’esperimento naturale presenta almeno tre problemi non superabili…
… First, causal density is very high, so sample size is critical, but many natural experiments have far too few data points… Second, a national society is holistically integrated; therefore, it is hard to get causal impermeability between the test and control groups. In the abortion-crime debate, for example, I indicated that a significant technical issue was how to account for the reality that people move between states… Third is the possibility of systematic, unobserved bias between the individuals or places that are subject to the treatment in the natural experiment as compared to those that are not. Consider the abortion-crime example. All kinds of plausible differences in political culture, social evolution, rational expectation for future challenges, and so on could vary between the early legalization states and the rest of the country… This is the irreducible problem for any such social natural experiment that does not use strict randomization for assignment to the test population, no matter how large the same size…
In sintesi: 1) campione ridotto 2) mancata impermeabilità 3) non casualità del gruppo di controllo.
Il povero Levitt ha preteso di indagare un mondo dove il batter d’ali di una farfalla puo’ causare cataclismi, un noto detto di cui val la pena rievocare l’origine…
… The actual event that inspired this observation was that, one day in 1961, Lorenz entered .506 instead of .506127 for one parameter in a climate-forecasting model and discovered that it produced a wildly different long-term weather forecast…
Spesso cio’ che sfugge agli accademici è ben presente a chi fa affari…
… Businesses are notoriously practical and results-oriented, and have sunk vast resources into trying to develop useful, reliable predictions for behavior in the absence of experiments. In doing so, they have run into the same problems and hit the same dead ends. I know, because I spent years doing it…
COMMENTO PERSONALE
L’obiezione alle critiche di Manzi viene facile: se non abbiamo in mano niente, allora meglio “qualcosa” che niente. Ma anche la controbiezione non è difficile: nessuno si presenta mai a mani vuote di fronte ad un problema. E comunque, quel che ci danno certe analisi è talmente poco che possiamo reperire altrove indizi più interessanti. Per esempio, nel caso di regressioni tipo quelle dei due casi presentati pesa più l’orientamento politico dei ricercatori che l’esito della ricerca.
