Superintelligence: Paths, Dangers, Strategies – Nick Bostrom
Argomento: avremo “superintelligenza” quando le macchine intelligenti sapranno costruire in breve tempo macchine più intelligenti di loro che sapranno costruire macchine più intelligenti di loro e così via all’infinito – trend: nel 2100 la famiglia media sarà 34 volte più ricca di oggi – IA sa replicare l’uomo pensante ma trova ostacoli nel fare cio’ che l’uomo fa senza pensare – perché gli scacchisti (e i pc che li battono) non sono particolarmente intelligenti – intelligenza generale e intelligenza specifica – quando arriverà IA? 10-50-80 –
Past developments and present capabilities
History, at the largest scale, seems to exhibit a sequence of distinct growth modes, each much more rapid than its predecessor. This pattern has been taken to suggest that another (even faster) growth mode might be possible.
Note: SINGOLARITA’ PRECEDENTI
Growth modes and big history
On a geological or even evolutionary timescale, the rise of Homo sapiens from our last common ancestor with the great apes happened swiftly. We developed upright posture, opposable thumbs, and—crucially—some relatively minor changes in brain size and neurological organization that led to a great leap in cognitive ability. As a consequence, humans can think abstractly, communicate complex thoughts, and culturally accumulate information
Note:HOMO SAPIENS… TEMPI GEOLOGICI
More people meant more ideas; greater densities meant that ideas could spread more readily and that some individuals could devote themselves to developing specialized skills. These developments increased the rate of growth of economic productivity and technological capacity.
Note:PIÙ POPOLAZIONE, PIU’ IDEE, PIU’ VELOCITA’ NELLO SVILUPPO
A few hundred thousand years ago, in early human (or hominid) prehistory, growth was so slow that it took on the order of one million years for human productive capacity to increase sufficiently to sustain an additional one million individuals living at subsistence level. By 5000 bc, following the Agricultural Revolution, the rate of growth had increased to the point where the same amount of growth took just two centuries. Today, following the Industrial Revolution, the world economy grows on average by that amount every ninety minutes.
Note:ACCELERAZIONE NELLA CRESCITA
If the world economy continues to grow at the same pace as it has over the past fifty years, then the world will be some 4.8 times richer by 2050 and about 34 times richer by 2100 than it is today.
Note:LA RICCHEZZA NEL 2100
If another such transition to a different growth mode were to occur, and it were of similar magnitude to the previous two, it would result in a new growth regime in which the world economy would double in size about every two weeks.
Note:E IL FUTURO?
The singularity-related idea that interests us here is the possibility of an intelligence explosion, particularly the prospect of machine superintelligence.
the case for taking seriously the prospect of a machine intelligence revolution need not rely on curve-fitting exercises or extrapolations from past economic growth.
Note:IL METODO DELL‘ESTRAPOLAZIONE
Machines matching humans in general intelligence—that is, possessing common sense and an effective ability to learn, reason, and plan to meet complex information-processing challenges across a wide range of natural and abstract domains—have been expected since the invention of computers in the 1940s. At that time, the advent of such machines was often placed some twenty years into the future.
Note:IA… ASPETTATIVE TRADITE
however many stops there are between here and human-level machine intelligence, the latter is not the final destination. The next stop, just a short distance farther along the tracks, is superhuman-level machine intelligence.
Note:NON IA MA SUPER IA
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
Note:DEFINIZIONE DI SUPER IA DA PARTE DI I J GOOD
In later decades, systems would be created that demonstrated that machines could compose music in the style of various classical composers, outperform junior doctors in certain clinical diagnostic tasks, drive cars autonomously, and make patentable inventions.19
The methods that produced successes in the early demonstration systems often proved difficult to extend to a wider variety of problems or to harder problem instances. One reason for this is the “combinatorial explosion” of possibilities that must be explored by methods that rely on something like exhaustive search.
To overcome the combinatorial explosion, one needs algorithms that exploit structure in the target domain and take advantage of prior knowledge by using heuristic search, planning, and flexible abstract representations
Note: L’OSTACOLO DELL’ESPLOSIONE COMBINATORIA
The performance of these early systems also suffered because of poor methods for handling uncertainty, reliance on brittle and ungrounded symbolic representations
Note:ALTRO OSTACOLO: L’INCERTEZZA
The ensuing years saw a great proliferation of expert systems. Designed as support tools for decision makers,
Note:SISTEMI ESPERTI... AFFIANCAMENTO
The newly popular techniques, which included neural networks and genetic algorithms, promised to overcome some of the shortcomings
one of the major theoretical developments of the past twenty years has been a clearer realization of how superficially disparate techniques can be understood as special cases within a common mathematical framework…This perspective allows neural nets to be compared with a larger class of algorithms for learning classifiers from examples—“decision trees,” “logistic regression models,” “support vector machines,” “naive Bayes,” “k-nearest-neighbors regression,” among others….
Note:CASI SPECIALI E VARIETÀ DI APPROCCIO
Accordingly, one can view artificial intelligence as a quest to find shortcuts: ways of tractably approximating the Bayesian ideal by sacrificing some optimality
Note:IA E L’OTTIMO
relating learning problems from specific domains to the general problem of Bayesian inference
Note:BAYES SEMPRE SULLO SFONDO
Artificial intelligence already outperforms human intelligence in many domains…AIs now beat human champions in a wide range of games…
Note:MACCHINE GIA’ SUPERIORI ALL’UOMO IN MOLTI CAMPI
It was once supposed, perhaps not unreasonably, that in order for a computer to play chess at grandmaster level, it would have to be endowed with a high degree of general intelligence…Not so. It turned out to be possible to build a perfectly fine chess engine around a special-purpose algorithm.40 When implemented on the fast processors that became available towards the end of the twentieth century, it produces very strong play…
Note:SCACCHI IQ E INTELLIGENZA SPECIFICA
Donald Knuth was struck that “AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do ‘without thinking’—that, somehow, is much harder!”
Note:CIÒ CHE SI FA SENZA PENSARE
Common sense and natural language understanding have also turned out to be difficult.
Chess-playing expertise turned out to be achievable by means of a surprisingly simple algorithm. It is tempting to speculate that other capabilities—such as general reasoning ability, or some key ability involved in programming—might likewise be achievable through some surprisingly simple algorithm. The fact that the best performance at one time is attained through a complicated mechanism does not mean that no simple mechanism could do the job as well or better.
Note:SEMPLICITÀ E SCORCIATOIE… LA SPERANZA IA
Now, it must be stressed that the demarcation between artificial intelligence and software in general is not sharp… though this brings us back to McCarthy’s dictum that when something works it is no longer called AI. A more relevant distinction for our purposes is that between systems that have a narrow range of cognitive capability (whether they be called “AI” or not) and systems that have more generally applicable problem-solving capacities…
One high-stakes and extremely competitive environment in which AI systems operate today is the global financial market.
Note:MERCATI FINANZIARI… UN AMBITO GIA’ DOMINATO DALLE MACCHINE
Opinions about the future of machine intelligence
Nils Nilsson, one of the old-timers in the field, complains that his present-day colleagues lack the boldness of spirit that propelled the pioneers of his own generation…Nilsson’s sentiment has been echoed by several others of the founders, including Marvin Minsky, John McCarthy, and Patrick Winston….
Note:MANCA L’AUDACIA DI UN TEMPO
Expert opinions about the future of AI vary wildly. There is disagreement about timescales as well as about what forms AI might eventually take.
The combined sample gave the following (median) estimate: 10% probability of HLMI by 2022, 50% probability by 2040, and 90% probability by 2075.
My own view is that the median numbers reported in the expert survey do not have enough probability mass on later arrival dates. A 10% probability of HLMI not having been developed by 2075 or even 2100 (after conditionalizing on “human scientific activity continuing without major negative disruption”) seems too low.
Note:I NUMERI ANDREBBERO ALZATI
Historically, AI researchers have not had a strong record of being able to predict the rate of advances in their own field or the shape that such advances would take.
Note:RECORD TRACK MOLTO SCARSO
Small sample sizes, selection biases, and—above all—the inherent unreliability of the subjective opinions elicited mean that one should not read too much into these expert surveys and interviews.