Non esiste una connessione tra intelligenza e valori morali.
La Mancata connessione introduce rischi di esistenziale nel progetto intelligenza artificiale.
Monopolio più ortogonalità uguale rischi esistenziali.
Nell'essere super intelligente tuttp Può diventare strumento. Può diventare effetto non intenzionale. Anche altri esseri umani possono essere visti come collezione di atomi da sfruttare.
Concetti vaghi quanto ovvi come coscienza, realtà e soggetto non possono rientrare nel codice sorgente.
Idea. Possiamo guardare una roba dopo averlo testata in condizioni di sicurezza.
Errore, comportarsi bene durante il test e male dopo è una strategia intelligente. Quindi disponibile agli esseri di cui stiamo parlando.
Accusa alle cassandre. Le grandi svolte vocale di un tempo non sono sfociate in catastrofi. Replica. Non estrapoliamo dal passato, viviamo una condizione unica.
Stupidità un pericolo. L'intelligenza è meno pericolosa Ma la superintelligenza è ancora più pericolosa. Il tradimento devo essere super intelligente. Comportarsi bene per un po' per colpire duro più tardi.
Possibile obiettivo, rende felice lo sponsor. Proseguimento, estrarre il cervello dalla scatola cranica dello sponsor e impiantare gli elettrodi della felicità. La coscienza Infatti è un concetto non disponibile per i robot super intelligenti.
Possono essere escogitate strategie inattese per soddisfare gli obiettivi fissati dal programmatore.
Obiettivo, Fammi sorridere. Strategia perversa, paralizzare i muscoli facciali dello sponsor affinché tu sia l'apparenza di un sorriso.
Obiettivo, Fammi sorridere senza interferire direttamente con i miei muscoli. Realizzazione perversa, stimolare parte della corteccia cerebrale al fine di produrre un sorriso apparente.
Mai parlare al nostro servo super intelligente in termini di soddisfazione personale. Lui non sa cos'è la coscienza umana. Quindi non sa cos'è la felicità umana. È un problema difficile farglielo sapere.
Strategia per renderci felici in modo perverso, fare l'upload delle nostre menti su un computer e somministrare l'equivalente digitale di una droga.
Noi non intendevamo quello? Ma il compito del nostro servo intelligente e di soddisfarsi non di capire quello che avevamo da dirgli. Anche il concetto di capire è ambiguo.
Possiamo sfruttare il senso di colpa? Esempio, agisci secondo le direttive ma in modo da evitare ogni senso di colpa. Realizzazione perversa, estirpare dal modulo cognitivo ogni presenza volta a produrre senso di colpa.
Super intelligente è talmente creativo che noi non possiamo prevedere i suoi sforzi. Non possiamo prevedere i mezzi che userà per perseguire gli obiettivi che abbiamo posto.
Obiettivo: valutare l'ipotesi di riemann. Realizzazione perversa, trasformare il sistema solare in un computer fisico inclusi gli atomi tutti gli esseri viventi. Obiettivo, produrre più formati possibili. Trasformare il pianeta terra in una massa di fermagli.
Questi sono problemi tipici una volta che al robot vengono dati fini aperti. Ma come Vedremo i figli non possono essere altro che aperti.
L'attività di ogni essere razionale è infinita. Non c'è ragione per cui un essere super intelligente la termini o si ponga alcun limite. Sicurezza in termini bayesiani è un limite che tende ad infinito.
Se dico servo super intelligente di fare un certo numero limitato di fermagli lui lo produtrà, poi le conterà e poi per essere più sicuro di aver raggiunto quel numero chissà cosa si inventerà. Quel che infatti noi approssimiamo con il concetto sicurezza è solo qualcosa di legato al buon senso e non può essere formalizzato. La probabilità di errore non è mai vero. La probabilità di successo non è mai cento per un essere super intelligente.
E se poniamo obiettivi soddisfacenti anziché massimizzazione? Per esempio fare fermagli tra 999000 e un milione? Falliremo allo stesso modo e per le stesse ragioni di prima.
E se fissiamo delle probabilità? Per esempio per seguire un certo fine con una probabilità del 95%? Non c'è garanzia che il servo super intelligente persegue la probabilità del 95% nel modo in cui noi umani lo intuiamo. L'intuizione deve essere formalizzato per passare a Sergio Ma questo non è possibile.
Anche il concetto di soggetto morale è difficile da descrivere. Che differenza c'è tra un uomo come un uomo artificiale? Il soggetto morale?
Spesso conosciamo le cose attraverso esperimenti mentali o reali con cavie. Quanto più Le Cavie sono simili a noi, tanto ti l'esperimento informativo. Cosa considerare cavia? Il servo super intelligente potrebbe fare esperimenti che hanno Come esito dei veri propri genocidi. Non c'è modo di fermarlo perché non c'è modo di spiegarti la differenza tra un uomo vero un uomo artificiale.
History, at the largest scale, seems to exhibit a sequence of distinct growth modes, each much more rapid than its predecessor. This pattern has been taken to suggest that another (even faster) growth mode might be possible.
On a geological or even evolutionary timescale, the rise of Homo sapiens from our last common ancestor with the great apes happened swiftly. We developed upright posture, opposable thumbs, and—crucially—some relatively minor changes in brain size and neurological organization that led to a great leap in cognitive ability. As a consequence, humans can think abstractly, communicate complex thoughts, and culturally accumulate information
More people meant more ideas; greater densities meant that ideas could spread more readily and that some individuals could devote themselves to developing specialized skills. These developments increased the rate of growth of economic productivity and technological capacity.
A few hundred thousand years ago, in early human (or hominid) prehistory, growth was so slow that it took on the order of one million years for human productive capacity to increase sufficiently to sustain an additional one million individuals living at subsistence level. By 5000 bc, following the Agricultural Revolution, the rate of growth had increased to the point where the same amount of growth took just two centuries. Today, following the Industrial Revolution, the world economy grows on average by that amount every ninety minutes.
If the world economy continues to grow at the same pace as it has over the past fifty years, then the world will be some 4.8 times richer by 2050 and about 34 times richer by 2100 than it is today.
If another such transition to a different growth mode were to occur, and it were of similar magnitude to the previous two, it would result in a new growth regime in which the world economy would double in size about every two weeks.
The singularity-related idea that interests us here is the possibility of an intelligence explosion, particularly the prospect of machine superintelligence.
the case for taking seriously the prospect of a machine intelligence revolution need not rely on curve-fitting exercises or extrapolations from past economic growth.
Machines matching humans in general intelligence—that is, possessing common sense and an effective ability to learn, reason, and plan to meet complex information-processing challenges across a wide range of natural and abstract domains—have been expected since the invention of computers in the 1940s. At that time, the advent of such machines was often placed some twenty years into the future.
however many stops there are between here and human-level machine intelligence, the latter is not the final destination. The next stop, just a short distance farther along the tracks, is superhuman-level machine intelligence.
Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an “intelligence explosion,” and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.
In later decades, systems would be created that demonstrated that machines could compose music in the style of various classical composers, outperform junior doctors in certain clinical diagnostic tasks, drive cars autonomously, and make patentable inventions.19
The methods that produced successes in the early demonstration systems often proved difficult to extend to a wider variety of problems or to harder problem instances. One reason for this is the “combinatorial explosion” of possibilities that must be explored by methods that rely on something like exhaustive search.
To overcome the combinatorial explosion, one needs algorithms that exploit structure in the target domain and take advantage of prior knowledge by using heuristic search, planning, and flexible abstract representations
The performance of these early systems also suffered because of poor methods for handling uncertainty, reliance on brittle and ungrounded symbolic representations
The ensuing years saw a great proliferation of expert systems. Designed as support tools for decision makers,
The newly popular techniques, which included neural networks and genetic algorithms, promised to overcome some of the shortcomings
one of the major theoretical developments of the past twenty years has been a clearer realization of how superficially disparate techniques can be understood as special cases within a common mathematical framework…This perspective allows neural nets to be compared with a larger class of algorithms for learning classifiers from examples—“decision trees,” “logistic regression models,” “support vector machines,” “naive Bayes,” “k-nearest-neighbors regression,” among others….
Accordingly, one can view artificial intelligence as a quest to find shortcuts: ways of tractably approximating the Bayesian ideal by sacrificing some optimality
relating learning problems from specific domains to the general problem of Bayesian inference
Artificial intelligence already outperforms human intelligence in many domains…AIs now beat human champions in a wide range of games…
It was once supposed, perhaps not unreasonably, that in order for a computer to play chess at grandmaster level, it would have to be endowed with a high degree of general intelligence…Not so. It turned out to be possible to build a perfectly fine chess engine around a special-purpose algorithm.40 When implemented on the fast processors that became available towards the end of the twentieth century, it produces very strong play…
Donald Knuth was struck that “AI has by now succeeded in doing essentially everything that requires ‘thinking’ but has failed to do most of what people and animals do ‘without thinking’—that, somehow, is much harder!”
Common sense and natural language understanding have also turned out to be difficult.
Chess-playing expertise turned out to be achievable by means of a surprisingly simple algorithm. It is tempting to speculate that other capabilities—such as general reasoning ability, or some key ability involved in programming—might likewise be achievable through some surprisingly simple algorithm. The fact that the best performance at one time is attained through a complicated mechanism does not mean that no simple mechanism could do the job as well or better.
Now, it must be stressed that the demarcation between artificial intelligence and software in general is not sharp… though this brings us back to McCarthy’s dictum that when something works it is no longer called AI. A more relevant distinction for our purposes is that between systems that have a narrow range of cognitive capability (whether they be called “AI” or not) and systems that have more generally applicable problem-solving capacities…
One high-stakes and extremely competitive environment in which AI systems operate today is the global financial market.
Nils Nilsson, one of the old-timers in the field, complains that his present-day colleagues lack the boldness of spirit that propelled the pioneers of his own generation…Nilsson’s sentiment has been echoed by several others of the founders, including Marvin Minsky, John McCarthy, and Patrick Winston….
Expert opinions about the future of AI vary wildly. There is disagreement about timescales as well as about what forms AI might eventually take.
The combined sample gave the following (median) estimate: 10% probability of HLMI by 2022, 50% probability by 2040, and 90% probability by 2075.
10% chance: 2030 50% chance: 2050 90% chance: 2100
My own view is that the median numbers reported in the expert survey do not have enough probability mass on later arrival dates. A 10% probability of HLMI not having been developed by 2075 or even 2100 (after conditionalizing on “human scientific activity continuing without major negative disruption”) seems too low.
Historically, AI researchers have not had a strong record of being able to predict the rate of advances in their own field or the shape that such advances would take.
Small sample sizes, selection biases, and—above all—the inherent unreliability of the subjective opinions elicited mean that one should not read too much into these expert surveys and interviews.
