SCHIAVI DEI ROBOT

SCHIAVI DEI ROBOT

Secondo chi studia queste cose a tempo pieno, i robot intelligenti suscitano due paure:

1) FOOM. Quando il primo robot intelligente saprà costruire un robot più intelligente di lui che saprà costruire un robot più intelligente di lui, l'intelligenza esploderà e il robot monopolista conquisterà il mondo. Questo potrebbe avvenire nel corso di un week-end.

1) AGENCY. Demanderemo molti compiti a robot sempre più intelligenti, i quali ci inganneranno eludendo i nostri controlli. L'intelligenza è soprattutto capacità di ingannare: i bambini più intelligenti, ricordiamolo, sono quelli che dicono più bugie.

Il primo è un classico problema legato alla crescita innovativa, il secondo un classico problema legato ai costi di agenzia.

Ebbene, esiste una vasta e rassicurante letteratura economica sia sul primo tema che sul secondo: storicamente, la stragrande maggioranza dell'innovazione è stata lenta, incrementale e distribuita in settori e luoghi differenziati, difficilmente si diffonde attraverso monopoli; l'azzardo morale puo' essere controllato.

Certo, chi sostiene "questa volta sarà diverso" non dà valore ai precedenti. Io sì.

http://www.overcomingbias.com/2019/04/agency-failure-ai-apocalypse.html

L’esplosione dell’intelligenza

Superintelligence: Paths, Dangers, Strategies – Nick Bostrom

***

Argomento: avremo “superintelligenza” quando le macchine intelligenti sapranno costruire in breve tempo macchine più intelligenti di loro che sapranno costruire macchine più intelligenti di loro e così via all’infinito –  trend: nel 2100 la famiglia media sarà 34 volte più ricca di oggi – IA sa replicare l’uomo pensante ma trova ostacoli nel fare cio’ che l’uomo fa senza pensare –  perché gli scacchisti (e i pc che li battono) non sono particolarmente intelligenti – intelligenza generale e intelligenza specifica – quando arriverà IA? 10-50-80 –

***

Past developments and present capabilities

History, at the largest scale, seems to exhibit a sequence of distinct growth modes, each much more rapid than its predecessor. This pattern has been taken to suggest that another (even faster) growth mode might be possible.

Note: SINGOLARITA’ PRECEDENTI

Growth modes and big history

On a geological or even evolutionary timescale, the rise of Homo sapiens from our last common ancestor with the great apes happened swiftly. We developed upright posture, opposable thumbs, and—crucially—some relatively minor changes in brain size and neurological organization that led to a great leap in cognitive ability. As a consequence, humans can think abstractly, communicate complex thoughts, and culturally accumulate information

Note:HOMO SAPIENS… TEMPI GEOLOGICI

More people meant more ideas; greater densities meant that ideas could spread more readily and that some individuals could devote themselves to developing specialized skills. These developments increased the rate of growth of economic productivity and technological capacity.

Note:PIÙ POPOLAZIONE, PIU’ IDEE, PIU’ VELOCITA’ NELLO SVILUPPO

A few hundred thousand years ago, in early human (or hominid) prehistory, growth was so slow that it took on the order of one million years for human productive capacity to increase sufficiently to sustain an additional one million individuals living at subsistence level. By 5000 bc, following the Agricultural Revolution, the rate of growth had increased to the point where the same amount of growth took just two centuries. Today, following the Industrial Revolution, the world economy grows on average by that amount every ninety minutes.

Note:ACCELERAZIONE NELLA CRESCITA

If the world economy continues to grow at the same pace as it has over the past fifty years, then the world will be some 4.8 times richer by 2050 and about 34 times richer by 2100 than it is today.

Note:LA RICCHEZZA NEL 2100

If another such transition to a different growth mode were to occur, and it were of similar magnitude to the previous two, it would result in a new growth regime in which the world economy would double in size about every two weeks.

Note:E IL FUTURO?

The singularity-related idea that interests us here is the possibility of an intelligence explosion, particularly the prospect of machine superintelligence.

Note:SUPERINTELLIGENZA

the case for taking seriously the prospect of a machine intelligence revolution need not rely on curve-fitting exercises or extrapolations from past economic growth.

Note:IL METODO DELL‘ESTRAPOLAZIONE

Great expectations

Machines matching humans in general intelligence—that is, possessing common sense and an effective ability to learn, reason, and plan to meet complex information-processing challenges across a wide range of natural and abstract domains—have been expected since the invention of computers in the 1940s. At that time, the advent of such machines was often placed some twenty years into the future.

Note:IA… ASPETTATIVE TRADITE

however many stops there are between here and human-level machine intelligence, the latter is not the final destination. The next stop, just a short distance farther along the tracks, is superhuman-level machine intelligence.

Note:NON IA MA SUPER IA

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.

Note:DEFINIZIONE DI SUPER IA DA PARTE DI I J GOOD

In later decades, systems would be created that demonstrated that machines could compose music in the style of various classical composers, outperform junior doctors in certain clinical diagnostic tasks, drive cars autonomously, and make patentable inventions.19

Note:AD OGGI

The methods that produced successes in the early demonstration systems often proved difficult to extend to a wider variety of problems or to harder problem instances. One reason for this is the “combinatorial explosion” of possibilities that must be explored by methods that rely on something like exhaustive search.

Note:MA…

To overcome the combinatorial explosion, one needs algorithms that exploit structure in the target domain and take advantage of prior knowledge by using heuristic search, planning, and flexible abstract representations

Note: L’OSTACOLO DELL’ESPLOSIONE COMBINATORIA

The performance of these early systems also suffered because of poor methods for handling uncertainty, reliance on brittle and ungrounded symbolic representations

Note:ALTRO OSTACOLO: L’INCERTEZZA

The ensuing years saw a great proliferation of expert systems. Designed as support tools for decision makers,

Note:SISTEMI ESPERTI... AFFIANCAMENTO

The newly popular techniques, which included neural networks and genetic algorithms, promised to overcome some of the shortcomings

Note:NUOVE TECNICHE

one of the major theoretical developments of the past twenty years has been a clearer realization of how superficially disparate techniques can be understood as special cases within a common mathematical framework…This perspective allows neural nets to be compared with a larger class of algorithms for learning classifiers from examples—“decision trees,” “logistic regression models,” “support vector machines,” “naive Bayes,” “k-nearest-neighbors regression,” among others….

Note:CASI SPECIALI E VARIETÀ DI APPROCCIO

Accordingly, one can view artificial intelligence as a quest to find shortcuts: ways of tractably approximating the Bayesian ideal by sacrificing some optimality

Note:IA E L’OTTIMO

relating learning problems from specific domains to the general problem of Bayesian inference

Note:BAYES SEMPRE  SULLO SFONDO

Artificial intelligence already outperforms human intelligence in many domains…AIs now beat human champions in a wide range of games…

Note:MACCHINE GIA’ SUPERIORI ALL’UOMO IN MOLTI CAMPI

It was once supposed, perhaps not unreasonably, that in order for a computer to play chess at grandmaster level, it would have to be endowed with a high degree of general intelligence…Not so. It turned out to be possible to build a perfectly fine chess engine around a special-purpose algorithm.40 When implemented on the fast processors that became available towards the end of the twentieth century, it produces very strong play…

Note:SCACCHI IQ E INTELLIGENZA SPECIFICA

Donald Knuth was struck that “AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do ‘without thinking’—that, somehow, is much harder!”

Note:CIÒ CHE SI FA SENZA PENSARE

Common sense and natural language understanding have also turned out to be difficult.

Note:SENSO COMUNE

Chess-playing expertise turned out to be achievable by means of a surprisingly simple algorithm. It is tempting to speculate that other capabilities—such as general reasoning ability, or some key ability involved in programming—might likewise be achievable through some surprisingly simple algorithm. The fact that the best performance at one time is attained through a complicated mechanism does not mean that no simple mechanism could do the job as well or better.

Note:SEMPLICITÀ E SCORCIATOIE… LA SPERANZA IA

Now, it must be stressed that the demarcation between artificial intelligence and software in general is not sharp… though this brings us back to McCarthy’s dictum that when something works it is no longer called AI. A more relevant distinction for our purposes is that between systems that have a narrow range of cognitive capability (whether they be called “AI” or not) and systems that have more generally applicable problem-solving capacities…

NoteTASSONOMIA

One high-stakes and extremely competitive environment in which AI systems operate today is the global financial market.

Note:MERCATI FINANZIARI… UN AMBITO GIA’ DOMINATO DALLE MACCHINE

Opinions about the future of machine intelligence

Nils Nilsson, one of the old-timers in the field, complains that his present-day colleagues lack the boldness of spirit that propelled the pioneers of his own generation…Nilsson’s sentiment has been echoed by several others of the founders, including Marvin Minsky, John McCarthy, and Patrick Winston….

Note:MANCA L’AUDACIA DI UN TEMPO

Expert opinions about the future of AI vary wildly. There is disagreement about timescales as well as about what forms AI might eventually take.

Note:DISACCORDO

The combined sample gave the following (median) estimate: 10% probability of HLMI by 2022, 50% probability by 2040, and 90% probability by 2075.

Note:SONDAGGIO TRA ESPERTI

10% chance: 2030 50% chance: 2050 90% chance: 2100

Note:NILS NILSSON

My own view is that the median numbers reported in the expert survey do not have enough probability mass on later arrival dates. A 10% probability of HLMI not having been developed by 2075 or even 2100 (after conditionalizing on “human scientific activity continuing without major negative disruption”) seems too low.

Note:I NUMERI ANDREBBERO ALZATI

Historically, AI researchers have not had a strong record of being able to predict the rate of advances in their own field or the shape that such advances would take.

Note:RECORD TRACK MOLTO SCARSO

Small sample sizes, selection biases, and—above all—the inherent unreliability of the subjective opinions elicited mean that one should not read too much into these expert surveys and interviews.

PROBLEMI DELLO STRUMENTO SONDAGGISTICO

I trafficoni dell’AI

I trafficoni dell’AI

The Hanson-Yudkowsky AI-Foom Debate – Robin Hanson and Eliezer Yudkowsky

***

Trigger warning: domani potranno essere costruiti robot più intelligenti di noi che sapranno costruire robot più intelligenti di loro. Come finirà questa storia? – emulatori o robot? – analogia o modelli? – due stili cognitivi per pensare al futuro lontano – il debole legame tra scienza e innovazione – tre modelli di innovazione radicale a cui ispirarsi per le analogia: nascita dell’homo sapiens, nascita dell’agricoltura, nascita dell’industria –

***

Chapter 1 Fund UberTool?

Sometimes a set of tool types will stumble into conditions especially favorable for mutual improvement.

Note:IL MIGLIORAMENTO RECIPROCO: MEZZI CHE COSTRUISCONO ALTRI MEZZI… E LA CRESCITA ESPLODE

Such favorable storms of mutual improvementusually run out quickly, however, and in all of human history no more than three storms have had a large and sustained enough impact to substantially change world economic growth rates.

Note:STORIA UMANA: TRE TEMPESTE A CUI ISPIRARSI PER PREVEDERE

Imagine you are a venture capitalist reviewing a proposed business plan. UberTool Corp has identified a candidate set of mutually aiding tools, and plans to spend millions pushing those tools through a mutual improvement storm.

Note:SCENARIO: SE UN’ IMPRESA CONQUISTASSE IL MONDO

UberTool does not plan to stop their closed self-improvement process until they are in a position to suddenly burst out and basically “take over the world.” … Now given such enormous potential gains, even a very tiny probability that UberTool could do what they planned might entice you to invest in them.

Note:PROBABILITA’ DI INVESTIMENTO NELL’AI FOOM: ELEVATE

Chapter 2 Engelbart as UberTool? Un antesignano

Yesterday I described UberTool, an imaginary company planning to push a set of tools through a mutual-improvement process;

Note:L’ IDEA DEI PC CHE COSTRUISCONO PC.

Augmenting Human Intellect: A Conceptual Framework … He understood not just that computer tools were especially open to mutual improvement… [Engelbart] is best known for inventing the computer mouse … Now to his credit, Doug never suggested that his team, even if better funded, might advance so far so fast as to “take over the world.” … Doug Engelbart understood what few others did—not just that computers could enable fantastic especially-mutually-improving tools, but lots of detail about what those tools would look like. …

Note:DOUG CONQUISTERÀ IL MONDO?

Chapter 3 Friendly Teams

Just as humans displaced chimps, farmers displaced hunters, and industry displaced farming, would a group with this much of a head start on such a general better tech have a decent shot at displacing industry folks? And if so, shouldn’t the rest of the world have worried about how “friendly” they were?

Note:LA CATENA DEGLI SPIAZZAMENTI. DOBBIAMO TEMERE IL VINCITORE?

In fact, while Engelbart’s ideas had important legacies, his team didn’t come remotely close to displacing much of anything. He lost most of his funding in the early 1970s, and his team dispersed.

Note:IL FALLIMENTO DI ENGELBART

But what makes that scenario reasonable if the UberTool scenario is not?

Note:LA BRUTTA FINE DI ENGELBART CI RASSICURA SUL MONOPOLISTA CATTIVO?

Chapter 4 Friendliness Factors

how much better will the best firm be relative to the average, second best, or worst?

Note:TENDENZA AL MONOPOLIO

Here are a few factors: …Resource Variance—The more competitors vary in resources, the more performance varies. … Cumulative Advantage—The more prior wins help one win again, … Lumpy Design—The more quality depends on a few crucial choices, relative to many small choices, the more quality varies. … Interdependence—When firms need inputs from each other, … Info Leaks—The more info competitors can gain about others’ efforts, the more the best will be copied, reducing variance. … Legal Barriers… Anti-Trust… Network Effects—Users may prefer to use the same product regardless of its quality. …

Note:DA COSA DIPENDE L’ESISTENZA DEL MONOPOLIO?

Some key innovations in history were associated with very high variance in competitor success. For example, our form of life seems to have eliminated all trace of any other forms on Earth.

Note:I PRECEDENTI. L’ HOMO SAPIENS SEMBRA DOMINARE INCONTRASTATO

On the other hand, farming and industry innovations were associated with much less variance.

Note:AGRICOLTURA E INDUSTRIA

attribute this mainly to info becoming much leakier, in part due to more shared standards,

CI SALVERÀ LA FLUIDITÀ DELL INFO?

If you worry that one competitor will severely dominate all others in the next really big innovation, forcing you to worry about its “friendliness,” you should want to promote factors that reduce success variance.

Note:CONSIGLIO POLITICO

Chapter 6 Setting the Stage (come ragionare per prevedere: analisi o analogie?)

We seem to agree that: … Feasible approaches include direct hand-coding, based on a few big and lots of little insights, and on emulations of real human brains.

Note:LE DUE VIE VERSO LA IA: 1 PROGRAMMAZIONE 2 EMULAZIONE DEL CERVELLO UMANO

Machine intelligence will, more likely than not, appear within a century,

Note:ENTRO UN SECOLO. PROB. SUP. 50%

Math and deep insights (especially probability) can be powerful relative to trend fitting and crude analogies.

Note:MATH, PROBABILITÀ E ANALOGIE

Some should be thinking about how to create “friendly” machine intelligences.

Note:LA QUESTIONE CENTRALE

We seem to disagree modestly about the relative chances of the emulation and direct-coding approaches;

Note:IL DISACCORDO

Our largest disagreement seems to be on the chances that a single hand-coded version will suddenly and without warning change from nearly powerless to overwhelmingly powerful; I’d put it as less than 1% and he seems to put it as over 10%…. My style is more to apply standard methods and insights to unusual topics. So I accept at face value the apparent direct-coding progress to date, and the opinions of most old AI researchers …

Note:LO STILE INTUITIVO

putting apparently dissimilar events into relevantly similar categories. … These  suggest a single suddenly superpowerful AI is pretty unlikely.

Note:LO STILE ANALOGICO: PIU’ PROBABILE LA DIVERSITA’

Eliezer seems to instead rely on abstractions he has worked out for himself, not yet much adopted by a wider community of analysts, nor proven over a history of applications to diverse events.

Note:RAZIONALISMO SPECIFICO DELL’INGEGNERE

Chapter 8 Abstraction, Not Analogy

I’m not that happy with framing our analysis choices here as “surface analogies” versus “inside views.”

Note:SURFACE VS INSIDE VIEW… DISTINZIONE SVIANTE

More useful, I think, to see this as a choice of abstractions. An abstraction (Wikipedia) neglects some details to emphasize others.

Note:MEGLIO: ASTRAZIONE VS ANALISI SPECIFICA

For example, consider the oldest known tool, the hammer (Wikipedia). To understand how well an ordinary hammer performs its main function, we can abstract from details of shape and materials. To calculate the kinetic energy it delivers, we need only look at its length, head mass, and recoil energy percentage (given by its bending strength). …To see that it is not a good thing to throw at people, we can note it is heavy, hard, and sharp. To see that it is not a good thing to hold high in a lightning storm, we can note it is long and conducts electricity. To evaluate the cost to carry it around in a tool kit, we consider its volume and mass. … Whether something is “similar” to a hammer depends on whether it has similar relevant features. …

Note:ESEMPIO: IL MARTELLO

The issue is which abstractions are how useful for which purposes, not which features are “deep” vs. “surface.”

Note:LA QUESTIONE INFINE

The future story of the creation of designed minds must of course differ in exact details from everything that has gone before. But that does not mean that nothing before is informative about it.

Note:AI E LE ANALOGIE

Yes, when you struggle to identify relevant abstractions you may settle for analogizing… Analogies are bad not because they use “surface” features, but because the abstractions they use do not offer enough relevant insight for the purpose at hand.

Note:IL DIFETTO DELL’ ANALOGIA

I claim academic studies of innovation and economic growth offer relevant abstractions for understanding the future creation of machine minds,

Note:AI. ANALOGIE CON INNOVAZIONE E CRESCITA ECONOMICA

previous major transitions, such as humans, farming, and industry, are relevantly similar.

Note:ANALOGIE: HOMO SAPIENS AGRICOLTURA INDUSTRIA

You have previously said nothing is similar enough to this new event for analogy to be useful, so all we have is “causal modeling” (though you haven’t explained what you mean by this in this context). This post is a reply saying, no, there are more ways using abstractions; analogy and causal modeling are two particular ways to reason via abstractions, but there are many other ways.

Note:MODELLO CAUSALE… O INGEGNERISTICO

Everything is new to us at some point; we are always trying to make sense of new things by using the abstractions we have collected from trying to understand all the old things.

IL RADICALMENTE NUOVO

Chapter 10 AI Go Foom

hand-coded AI will come soon and in the form of a single suddenly superpowerful AI.

Note:L IPOTESI DI Y (LA MENO PROBABILE PER H)

A machine intelligence can directly rewrite its entire source code and redesign its entire physical hardware. While human brains can in principle modify themselves arbitrarily, in practice our limited understanding of ourselves means we mainly only change ourselves by thinking new thoughts. All else equal this means that machine brains have an advantage in improving themselves. …“object” vs. “meta” … It seems as if you think object ones don’t increase growth rates while meta ones do. …

Note:UN VANTAGGIO DELL’IPOTESI SOFTWARISTICA: LE MACCHINE SI RIPRODUCONO PIÙ EFFICACEMENTE. EUGENETICA INFORMATICA.

Chapter 12 Eliezer’s Meta-level Determinism (il ruolo della scienza nell’avanzamento umano)

it seems the basis for Eliezer’s claim that my analysis is untrustworthy “surface analogies” vs. his reliable “deep causes” is that, while I use long-vetted general social science understandings of factors influencing innovation, he uses his own new untested meta-level determinism theory.

Note:ANCORA ANALOGIA VS CAUSALITÀ

The last three strong transitions were humans, farming, and industry, and in terms of growth rate changes these seem to be of similar magnitude. Eliezer seems to predict we will discover the first of these was much stronger than the other two. And while the key causes of these transitions have long been hotly disputed, with many theories in play, Eliezer seems to pick specific winners for these disputes: intergenerational culture, writing, and scientific thinking.

Note:LE 3 TRANSIZIONI. IL SEGRETO DEL SUCCESSO UMANO

Few could write and what they wrote didn’t help farming much. Farming seems more plausibly to have resulted from a scale effect in the accumulation of innovations in abilities to manage plants and animals

Note:AGRICOLTURA E CUMULO DELLE CONOSCENZE. NO PENSIERO SCIENTIFICO

Also for industry, the key innovation does not seem to have been a scientific way of thinking—that popped up periodically in many times and places, and by itself wasn’t particularly useful. My guess is that the key was the formation of networks of science-like specialists, which wasn’t possible until the previous economy had reached a critical scale and density.

INDUSTRIA: NETWORK DI TRAFFICONI E MASSA CRITICA DEL CAPITALE. IL RUOLO DELLA SCIENZA E’ SECONDARIO.