I trafficoni dell’AI

I trafficoni dell’AI

The Hanson-Yudkowsky AI-Foom Debate – Robin Hanson and Eliezer Yudkowsky

***

Trigger warning: domani potranno essere costruiti robot più intelligenti di noi che sapranno costruire robot più intelligenti di loro. Come finirà questa storia? – emulatori o robot? – analogia o modelli? – due stili cognitivi per pensare al futuro lontano – il debole legame tra scienza e innovazione – tre modelli di innovazione radicale a cui ispirarsi per le analogia: nascita dell’homo sapiens, nascita dell’agricoltura, nascita dell’industria –

***

Chapter 1 Fund UberTool?

Sometimes a set of tool types will stumble into conditions especially favorable for mutual improvement.

Note:IL MIGLIORAMENTO RECIPROCO: MEZZI CHE COSTRUISCONO ALTRI MEZZI… E LA CRESCITA ESPLODE

Such favorable storms of mutual improvementusually run out quickly, however, and in all of human history no more than three storms have had a large and sustained enough impact to substantially change world economic growth rates.

Note:STORIA UMANA: TRE TEMPESTE A CUI ISPIRARSI PER PREVEDERE

Imagine you are a venture capitalist reviewing a proposed business plan. UberTool Corp has identified a candidate set of mutually aiding tools, and plans to spend millions pushing those tools through a mutual improvement storm.

Note:SCENARIO: SE UN’ IMPRESA CONQUISTASSE IL MONDO

UberTool does not plan to stop their closed self-improvement process until they are in a position to suddenly burst out and basically “take over the world.” … Now given such enormous potential gains, even a very tiny probability that UberTool could do what they planned might entice you to invest in them.

Note:PROBABILITA’ DI INVESTIMENTO NELL’AI FOOM: ELEVATE

Chapter 2 Engelbart as UberTool? Un antesignano

Yesterday I described UberTool, an imaginary company planning to push a set of tools through a mutual-improvement process;

Note:L’ IDEA DEI PC CHE COSTRUISCONO PC.

Augmenting Human Intellect: A Conceptual Framework … He understood not just that computer tools were especially open to mutual improvement… [Engelbart] is best known for inventing the computer mouse … Now to his credit, Doug never suggested that his team, even if better funded, might advance so far so fast as to “take over the world.” … Doug Engelbart understood what few others did—not just that computers could enable fantastic especially-mutually-improving tools, but lots of detail about what those tools would look like. …

Note:DOUG CONQUISTERÀ IL MONDO?

Chapter 3 Friendly Teams

Just as humans displaced chimps, farmers displaced hunters, and industry displaced farming, would a group with this much of a head start on such a general better tech have a decent shot at displacing industry folks? And if so, shouldn’t the rest of the world have worried about how “friendly” they were?

Note:LA CATENA DEGLI SPIAZZAMENTI. DOBBIAMO TEMERE IL VINCITORE?

In fact, while Engelbart’s ideas had important legacies, his team didn’t come remotely close to displacing much of anything. He lost most of his funding in the early 1970s, and his team dispersed.

Note:IL FALLIMENTO DI ENGELBART

But what makes that scenario reasonable if the UberTool scenario is not?

Note:LA BRUTTA FINE DI ENGELBART CI RASSICURA SUL MONOPOLISTA CATTIVO?

Chapter 4 Friendliness Factors

how much better will the best firm be relative to the average, second best, or worst?

Note:TENDENZA AL MONOPOLIO

Here are a few factors: …Resource Variance—The more competitors vary in resources, the more performance varies. … Cumulative Advantage—The more prior wins help one win again, … Lumpy Design—The more quality depends on a few crucial choices, relative to many small choices, the more quality varies. … Interdependence—When firms need inputs from each other, … Info Leaks—The more info competitors can gain about others’ efforts, the more the best will be copied, reducing variance. … Legal Barriers… Anti-Trust… Network Effects—Users may prefer to use the same product regardless of its quality. …

Note:DA COSA DIPENDE L’ESISTENZA DEL MONOPOLIO?

Some key innovations in history were associated with very high variance in competitor success. For example, our form of life seems to have eliminated all trace of any other forms on Earth.

Note:I PRECEDENTI. L’ HOMO SAPIENS SEMBRA DOMINARE INCONTRASTATO

On the other hand, farming and industry innovations were associated with much less variance.

Note:AGRICOLTURA E INDUSTRIA

attribute this mainly to info becoming much leakier, in part due to more shared standards,

CI SALVERÀ LA FLUIDITÀ DELL INFO?

If you worry that one competitor will severely dominate all others in the next really big innovation, forcing you to worry about its “friendliness,” you should want to promote factors that reduce success variance.

Note:CONSIGLIO POLITICO

Chapter 6 Setting the Stage (come ragionare per prevedere: analisi o analogie?)

We seem to agree that: … Feasible approaches include direct hand-coding, based on a few big and lots of little insights, and on emulations of real human brains.

Note:LE DUE VIE VERSO LA IA: 1 PROGRAMMAZIONE 2 EMULAZIONE DEL CERVELLO UMANO

Machine intelligence will, more likely than not, appear within a century,

Note:ENTRO UN SECOLO. PROB. SUP. 50%

Math and deep insights (especially probability) can be powerful relative to trend fitting and crude analogies.

Note:MATH, PROBABILITÀ E ANALOGIE

Some should be thinking about how to create “friendly” machine intelligences.

Note:LA QUESTIONE CENTRALE

We seem to disagree modestly about the relative chances of the emulation and direct-coding approaches;

Note:IL DISACCORDO

Our largest disagreement seems to be on the chances that a single hand-coded version will suddenly and without warning change from nearly powerless to overwhelmingly powerful; I’d put it as less than 1% and he seems to put it as over 10%…. My style is more to apply standard methods and insights to unusual topics. So I accept at face value the apparent direct-coding progress to date, and the opinions of most old AI researchers …

Note:LO STILE INTUITIVO

putting apparently dissimilar events into relevantly similar categories. … These  suggest a single suddenly superpowerful AI is pretty unlikely.

Note:LO STILE ANALOGICO: PIU’ PROBABILE LA DIVERSITA’

Eliezer seems to instead rely on abstractions he has worked out for himself, not yet much adopted by a wider community of analysts, nor proven over a history of applications to diverse events.

Note:RAZIONALISMO SPECIFICO DELL’INGEGNERE

Chapter 8 Abstraction, Not Analogy

I’m not that happy with framing our analysis choices here as “surface analogies” versus “inside views.”

Note:SURFACE VS INSIDE VIEW… DISTINZIONE SVIANTE

More useful, I think, to see this as a choice of abstractions. An abstraction (Wikipedia) neglects some details to emphasize others.

Note:MEGLIO: ASTRAZIONE VS ANALISI SPECIFICA

For example, consider the oldest known tool, the hammer (Wikipedia). To understand how well an ordinary hammer performs its main function, we can abstract from details of shape and materials. To calculate the kinetic energy it delivers, we need only look at its length, head mass, and recoil energy percentage (given by its bending strength). …To see that it is not a good thing to throw at people, we can note it is heavy, hard, and sharp. To see that it is not a good thing to hold high in a lightning storm, we can note it is long and conducts electricity. To evaluate the cost to carry it around in a tool kit, we consider its volume and mass. … Whether something is “similar” to a hammer depends on whether it has similar relevant features. …

Note:ESEMPIO: IL MARTELLO

The issue is which abstractions are how useful for which purposes, not which features are “deep” vs. “surface.”

Note:LA QUESTIONE INFINE

The future story of the creation of designed minds must of course differ in exact details from everything that has gone before. But that does not mean that nothing before is informative about it.

Note:AI E LE ANALOGIE

Yes, when you struggle to identify relevant abstractions you may settle for analogizing… Analogies are bad not because they use “surface” features, but because the abstractions they use do not offer enough relevant insight for the purpose at hand.

Note:IL DIFETTO DELL’ ANALOGIA

I claim academic studies of innovation and economic growth offer relevant abstractions for understanding the future creation of machine minds,

Note:AI. ANALOGIE CON INNOVAZIONE E CRESCITA ECONOMICA

previous major transitions, such as humans, farming, and industry, are relevantly similar.

Note:ANALOGIE: HOMO SAPIENS AGRICOLTURA INDUSTRIA

You have previously said nothing is similar enough to this new event for analogy to be useful, so all we have is “causal modeling” (though you haven’t explained what you mean by this in this context). This post is a reply saying, no, there are more ways using abstractions; analogy and causal modeling are two particular ways to reason via abstractions, but there are many other ways.

Note:MODELLO CAUSALE… O INGEGNERISTICO

Everything is new to us at some point; we are always trying to make sense of new things by using the abstractions we have collected from trying to understand all the old things.

IL RADICALMENTE NUOVO

Chapter 10 AI Go Foom

hand-coded AI will come soon and in the form of a single suddenly superpowerful AI.

Note:L IPOTESI DI Y (LA MENO PROBABILE PER H)

A machine intelligence can directly rewrite its entire source code and redesign its entire physical hardware. While human brains can in principle modify themselves arbitrarily, in practice our limited understanding of ourselves means we mainly only change ourselves by thinking new thoughts. All else equal this means that machine brains have an advantage in improving themselves. …“object” vs. “meta” … It seems as if you think object ones don’t increase growth rates while meta ones do. …

Note:UN VANTAGGIO DELL’IPOTESI SOFTWARISTICA: LE MACCHINE SI RIPRODUCONO PIÙ EFFICACEMENTE. EUGENETICA INFORMATICA.

Chapter 12 Eliezer’s Meta-level Determinism (il ruolo della scienza nell’avanzamento umano)

it seems the basis for Eliezer’s claim that my analysis is untrustworthy “surface analogies” vs. his reliable “deep causes” is that, while I use long-vetted general social science understandings of factors influencing innovation, he uses his own new untested meta-level determinism theory.

Note:ANCORA ANALOGIA VS CAUSALITÀ

The last three strong transitions were humans, farming, and industry, and in terms of growth rate changes these seem to be of similar magnitude. Eliezer seems to predict we will discover the first of these was much stronger than the other two. And while the key causes of these transitions have long been hotly disputed, with many theories in play, Eliezer seems to pick specific winners for these disputes: intergenerational culture, writing, and scientific thinking.

Note:LE 3 TRANSIZIONI. IL SEGRETO DEL SUCCESSO UMANO

Few could write and what they wrote didn’t help farming much. Farming seems more plausibly to have resulted from a scale effect in the accumulation of innovations in abilities to manage plants and animals

Note:AGRICOLTURA E CUMULO DELLE CONOSCENZE. NO PENSIERO SCIENTIFICO

Also for industry, the key innovation does not seem to have been a scientific way of thinking—that popped up periodically in many times and places, and by itself wasn’t particularly useful. My guess is that the key was the formation of networks of science-like specialists, which wasn’t possible until the previous economy had reached a critical scale and density.

INDUSTRIA: NETWORK DI TRAFFICONI E MASSA CRITICA DEL CAPITALE. IL RUOLO DELLA SCIENZA E’ SECONDARIO.

SAGGIO Le basi morali della politia

Le basi morali della politica

The Moral Foundations of Politics – The Righteous Mind: Why Good People are Divided by Politics and Religion – Jonathan Haidt

***

Trigger warning: – i cinque pilastri – perché vince la destra – equivoci sull’ “appello alla paura”

***

Care (hurting a child), Fairness (profiting from someone else’s undeserved loss), Loyalty (criticizing your nation to outsiders), Authority (disrespecting your father), and Sanctity (acting in a degrading or disgusting way).

Note:CINQUE PILASTRI DELLA MORALE

A NOTE ON INNATENESS

The brain is like a book, the first draft of which is written by the genes during fetal development. No chapters are complete at birth, and some are just rough outlines waiting to be filled in during childhood. But not a single chapter—be it on sexuality, language, food preferences, or morality—consists of blank pages

Note:IL CERVELLO È COME UN LIBRO

“Built-in” does not mean unmalleable; it means “organized in advance

Note:DEFINIZIONE DI INNATO

The list of five moral foundations was my first attempt to specify how the righteous mind was “organized in advance of experience.”

Note:MORALITÀ INNATA

1. THE CARE/HARM FOUNDATION

Mammals make fewer bets and invest a lot more in each one, so mammals face the challenge of caring for and nurturing their children for a long time.

Note:LA PROLE DEI MAMMIFERI

attachment theory, a well-supported theory that describes the system by which mothers and children regulate each other’s behavior so that the child gets a good mix of protection and opportunities for independent exploration.

Note:TEORIA DELL’ATTACCAMENTO

The moral matrix of liberals, in America and elsewhere, rests more heavily on the Care foundation than do the matrices of conservatives,

Note:LIBERAL E CURA

2. THE FAIRNESS/CHEATING FOUNDATION

Altruism toward kin is not a puzzle at all. Altruism toward non-kin, on the other hand, has presented one of the longest-running puzzles in the history of evolutionary thinking.

Note:ALTRUISMO

Trivers noted that evolution could create altruists in a species where individuals could remember their prior interactions with other individuals and then limit their current niceness to those who were likely to repay the favor.

Note:TRIVERS E LA CORRETTEZZA

Human life is a series of opportunities for mutually beneficial cooperation.

Note:COOPERAZIONE

On the left, fairness often implies equality, but on the right it means proportionality—people should be rewarded in proportion

Note:EGUAGLIANZA E PROPORZIONALITÀ

3. THE LOYALTY/BETRAYAL FOUNDATION

The male mind appears to be innately tribal—that is, structured in advance of experience so that boys and men enjoy doing the sorts of things that lead to group cohesion and success in conflicts between groups (including warfare).

Note:TRIBALISMI

The Loyalty/betrayal foundation is just a part of our innate preparation for meeting the adaptive challenge of forming cohesive coalitions.

Note:LEALTÀ E COESIONE TRIBALE

The love of loyal teammates is matched by a corresponding hatred of traitors, who are usually considered to be far worse than enemies.

Note:IL NEMICO

The left tends toward universalism and away from nationalism,26 so it often has trouble connecting to voters who rely on the Loyalty foundation.Indeed, because of its strong reliance upon the Care foundation, American liberals are often hostile to American foreign policy.

Note:I PROBLEMI DELLA SINISTRA CON LA LEALTÀ

4. THE AUTHORITY/SUBVERSION FOUNDATION

The urge to respect hierarchical relationships is so deep that many languages encode it directly. In French, as in other romance languages, speakers are forced to choose whether they’ll address someone using the respectful form (vous) or the familiar form (tu).

Note:GERARCHIA E LINGUAGGIO

The obvious way to begin thinking about the evolution of the Authority foundation is to consider the pecking orders and dominance hierarchies of chickens, dogs, chimpanzees,

Note:ALTRE SPECIE

As the primatologist Frans de Waal puts it: “Without agreement on rank and a certain respect for authority there can be no great sensitivity to social rules,

Note:GERARCHIA E SOCIALITÀ

When I began graduate school I subscribed to the common liberal belief that hierarchy = power = exploitation = evil.

Note:PROBLEMI A SINISTRA

5. THE SANCTITY/DEGRADATION FOUNDATION

Liberals score higher on measures of neophilia (also known as “openness to experience”), not just for new foods but also for new people, music, and ideas. Conservatives are higher on neophobia; they prefer to stick with what’s tried and true, and they care a lot more about guarding borders, boundaries, and traditions.

Note:NEOFILIA NEOFOBIA

The emotion of disgust evolved initially to optimize responses to the omnivore’s dilemma.39 Individuals who had a properly calibrated sense of disgust were able to consume more calories than their overly disgustable cousins while consuming fewer dangerous microbes than their insufficiently disgustable cousins.

Note:IL DISGUSTO DEGLI ONNIVORI

It’s a lot more effective to prevent infection by washing your food,

Note:LAVARE PULIRE PURIFICARE SANTIFICARE

The original adaptive challenge that drove the evolution of the Sanctity foundation, therefore, was the need to avoid pathogens, parasites,

Note:PARASSITI

Cultures differ in their attitudes toward immigrants, and there is some evidence that liberal and welcoming attitudes are more common in times and places where disease risks are lower.41 Plagues, epidemics, and new diseases are usually brought in by foreigners—as

Note:IMMIGRAZIONE

If we had no sense of disgust, I believe we would also have no sense of the sacred.

Note:SACRO E DISGUSTO

American conservatives are more likely to talk about “the sanctity of life” and “the sanctity of marriage.”

Note:DESTRA E SANTITÀ

On the left, however, the virtue of chastity is usually dismissed as outdated and sexist.

Note:ESEMPIO CASTITÀ

You can see the foundation’s original impurity-avoidance function in New Age grocery stores, where you’ll find a variety of products that promise to cleanse you of “toxins.”

Note:IL SACRO A SINISTRA: CIBO

If you dismiss the Sanctity foundation entirely, then it’s hard to understand the fuss over most of today’s biomedical controversies. The only ethical question about abortion becomes: At what point can a fetus feel pain?

Note:BIOETICA E SENSO DEL SACRO

The philosopher Leon Kass is among the foremost spokesmen for Shweder’s ethic of divinity, and for the Sanctity foundation on which it is based.

Note:KASS

Kass argued that our feelings of disgust can sometimes provide us with a valuable warning that we are going too far, even when we are morally dumbfounded and can’t justify those feelings by pointing to victims:

Note:RIPUGNANZA COME SEGNALE

The message of my talk to the Charlottesville Democrats was simple: Republicans understand moral psychology. Democrats don’t.Republicans have long understood that the elephant is in charge of political behavior, not the rider, and they know how elephants work.

Note:IL VANTAGGIO DELLA DESTRA NEL CAMP DELLA PSICOLOGIA SOCIALE

Republicans don’t just aim to cause fear, as some Democrats charge. They trigger the full range of intuitions described by Moral Foundations Theory.

LA PAURA NON C’ENTRA