10-14 The Hanson-Yudkowsky AI-Foom Debate by Robin Hanson, Eliezer Yudkowsky

10-14 The Hanson-Yudkowsky AI-Foom Debate by Robin Hanson, Eliezer Yudkowsky - eugeneticainformatica analogiavscausalità tretransizioni rivoluzioneindustrialeconnessionetrafficoni cumuloconoscienzenopensieroscientifico 

Chapter 10 AI Go FoomRead more at location 634

Note: Y: Ia sarà hand coded e concentrata Edit

Note: 10@@@@@@@@@@@@@@@@@@@ Edit

hand-coded AI will come soon and in the form of a single suddenly superpowerful AI.Read more at location 637

Note: L IPOTESI DI Y Edit

A machine intelligence can directly rewrite its entire source code and redesign its entire physical hardware. While human brains can in principle modify themselves arbitrarily, in practice our limited understanding of ourselves means we mainly only change ourselves by thinking new thoughts. All else equal this means that machine brains have an advantage in improving themselves.Read more at location 640

Note: LE MACCHINE SI RIPRODUCONO PIÙ EFFICACEMENTE? EUGENETICA INFORMATICA. Edit

“object” vs. “meta”Read more at location 674

Add a note

It seems as if you think object ones don’t increase growth rates while meta ones do.Read more at location 674

Add a note

Chapter 12 Eliezer’s Meta-level DeterminismRead more at location 816

Note: Il metodi alternativi: H: analogie con contesti simili e analisi dei precedenti E: conoscenza dei meccanismi specifici del campo oggetto Le 3 transizioni analoghe: uomo, agricoltura, industria Cause: nn tanto la conoscenza quanto la messa in rete delle singole conoscenze. E: l' agricoltura raccontata da un ottimizatore. H dà + peso al caso e quindi alle capacità combinatorie del sistema. E dà+ peso all' intelligenza e al progetto. Edit

Note: 12@@@@@@@@@@@@@@@@@@@ Edit

it seems the basis for Eliezer’s claim that my analysis is untrustworthy “surface analogies” vs. his reliable “deep causes” is that, while I use long-vetted general social science understandings of factors influencing innovation, he uses his own new untested meta-level determinism theory.Read more at location 826

Note: ANCORA ANALOGIA VS CAUSALITÀ Edit

The last three strong transitions were humans, farming, and industry, and in terms of growth rate changes these seem to be of similar magnitude. Eliezer seems to predict we will discover the first of these was much stronger than the other two. And while the key causes of these transitions have long been hotly disputed, with many theories in play, Eliezer seems to pick specific winners for these disputes: intergenerational culture, writing, and scientific thinking.Read more at location 866

Note: LE 3 TRANSIZIONI. IL SEGRETO DEL SUCCESSO UMANO Edit

Few could write and what they wrote didn’t help farming much. Farming seems more plausibly to have resulted from a scale effect in the accumulation of innovations in abilities to manage plants and animals—weRead more at location 871

Note: AGRICOLTURA E CUMULO DELLE CONOSCENZE. NO PENSIERO SCIENTIFICO Edit

Also for industry, the key innovation does not seem to have been a scientific way of thinking—that popped up periodically in many times and places, and by itself wasn’t particularly useful. My guess is that the key was the formation of networks of science-like specialists, which wasn’t possible until the previous economy had reached a critical scale and density.Read more at location 874

Note: INDUSTRIA: NETWORK DI TRAFFICONI E MASSA CRITICA DEL CAPITALE. NO PENSIERO SCIENTIFICO

1-10 The Hanson-Yudkowsky AI-Foom Debate by Robin Hanson, Eliezer Yudkowsky

1-10 The Hanson-Yudkowsky AI-Foom Debate by Robin Hanson, Eliezer Yudkowsky - letretempesteprecedenti homoagrindustr analogiaomodello? unuomoconquistailmondo? pcchecostruisconopc analogiaastrazione

The Hanson-Yudkowsky AI-Foom DebateRead more at location 3

Note: due approcci x prevedere: analogia vs. modello H è un economista evolutivo (hayekiano), E. è un ingegnere amante dei progetti che ottimizzano. Il primo è + empirista, il secondo + razionalista IA: emersione concentrata o diffusa? 2 IA possibili: hand coded e brain emulator Previsioni: esperto in scommesse (analista di precedenti) vs. esperto di settore (analista specifoco) effetto scala vs. abilità specifiche ** le diseguaglianze: il modo + efficiente x contenerle consiste nel lasciare una certa xmeabilità all informazoons (leak)Edit

Chapter 1 Fund UberTool?Read more at location 87

Note: 1@@@@@@@@@ @@@@@@@@ Edit

Sometimes a set of tool types will stumble into conditions especially favorable for mutual improvement.Read more at location 92

Note: LA CRESCITA ESPLODE Edit

Such favorable storms of mutual improvement usually run out quickly, however, and in all of human history no more than three storms have had a large and sustained enough impact to substantially change world economic growth rates.Read more at location 93

Note: STORIA UMANA: TRE TEMPESTE

Imagine you are a venture capitalist reviewing a proposed business plan. UberTool Corp has identified a candidate set of mutually aiding tools, and plans to spend millions pushing those tools through a mutual improvement storm.Read more at location 96

Note: SCENARIO: UN IMPRESA CONQUISTA IL MONDO Edit

UberTool does not plan to stop their closed self-improvement process until they are in a position to suddenly burst out and basically “take over the world.”Read more at location 101

Add a note

Now given such enormous potential gains, even a very tiny probability that UberTool could do what they planned might entice you to invest in them.Read more at location 103

Note: TINY PROBABILITY Edit

Chapter 2 Engelbart as UberTool?Read more at location 120

Note: 2@@@@@@@@@@ Edit

Yesterday I described UberTool, an imaginary company planning to push a set of tools through a mutual-improvement process;Read more at location 122

Add a note

Douglas EngelbartRead more at location 125

Note: L IDEA DEI PC CHE COSTRUISCONO PC. Edit

Human Intellect: A Conceptual FrameworkRead more at location 126

Add a note

He understood not just that computer tools were especially open to mutual improvement,Read more at location 128

Add a note

[Engelbart] is best known for inventing the computer mouseRead more at location 130

Add a note

Now to his credit, Doug never suggested that his team, even if better funded, might advance so far so fast as to “take over the world.”Read more at location 137

Note: DOUG CONQUISTERÀ IL MONDO Edit

Doug Engelbart understood what few others did—not just that computers could enable fantastic especially-mutually-improving tools, but lots of detail about what those tools would look like.Read more at location 140

Add a note

Chapter 3 Friendly TeamsRead more at location 157

Note: 3@@@@@@@@@@@ Edit

Just as humans displaced chimps, farmers displaced hunters, and industry displaced farming, would a group with this much of a head start on such a general better tech have a decent shot at displacing industry folks? And if so, shouldn’t the rest of the world have worried about how “friendly” they were?Read more at location 166

Note: LA CATENA DEGLI SPIAZZAMENTI. DOBBIAMO TEMERE IL VINCITORE? Edit

In fact, while Engelbart’s ideas had important legacies, his team didn’t come remotely close to displacing much of anything. He lost most of his funding in the early 1970s, and his team dispersed.Read more at location 169

Note: IL FALLIMENTO DI ENGELBART Edit

But what makes that scenario reasonable if the UberTool scenario is not?Read more at location 179

Note: LA BRUTTA FINE DI ENGELBART CI RASSICURA SUL MONOPOLISTA CATTIVO? Edit

Chapter 4 Friendliness FactorsRead more at location 192

Note: 4@@@@@@@@@@@ Edit

how much better will the best firm be relative to the average, second best, or worst?Read more at location 195

Note: TENDENZA AL MONOPOLIO Edit

Here are a few factors:Read more at location 196

Note: DA COSA DIPENDE IL MONOPOLIO? Edit

Resource Variance—The more competitors vary in resources, the more performance varies.Read more at location 197

Add a note

Cumulative Advantage—The more prior wins help one win again,Read more at location 198

Add a note

Lumpy Design—The more quality depends on a few crucial choices, relative to many small choices, the more quality varies.Read more at location 203

Add a note

Interdependence—When firms need inputs from each other,Read more at location 204

Add a note

Info Leaks—The more info competitors can gain about others’ efforts, the more the best will be copied, reducing variance.Read more at location 205

Add a note

Legal Barriers—MayRead more at location 208

Add a note

Anti-Trust—SocialRead more at location 209

Add a note

Network Effects—Users may prefer to use the same product regardless of its quality.Read more at location 213

Add a note

Some key innovations in history were associated with very high variance in competitor success. For example, our form of life seems to have eliminated all trace of any other forms on Earth.Read more at location 215

Note: I PRECEDENTI. LL HOMO SAPIENS SEMBRA DOMINARE INCONTRASTATO Edit

On the other hand, farming and industry innovations were associated with much less variance.Read more at location 216

Note: AGRICOLTURA E INDUSTRIA Edit

attribute this mainly to info becoming much leakier, in part due to more shared standards,Read more at location 218

Note: CI SALVERÀ LA FLUIDITÀ DELL INFO? Edit

If you worry that one competitor will severely dominate all others in the next really big innovation, forcing you to worry about its “friendliness,” you should want to promote factors that reduce success variance.Read more at location 219

Note: CONSIGLIO POLITICO Edit

Chapter 6 Setting the StageRead more at location 360

Note: 6@@@@@@@@@ Edit

We seem to agree that:Read more at location 363

Note: RIASSUNTO DEI TEMI Edit

Feasible approaches include direct hand-coding, based on a few big and lots of little insights, and on emulations of real human brains.Read more at location 364

Note: LE DUE VIE VERSO L IA: 1 PROGRAMMAZIONE 2 EMUAZINE DEL CERVELLO UMANO Edit

Machine intelligence will, more likely than not, appear within a century,Read more at location 365

Note: ENTRO UN SECOLO. PROB. SUP. 50% Edit

Math and deep insights (especially probability) can be powerful relative to trend fitting and crude analogies.Read more at location 368

Note: MATH PROBABILITÀ E ANALOGIE Edit

Some should be thinking about how to create “friendly” machine intelligences.Read more at location 370

Note: LA QUESTIONE CENTRALE Edit

We seem to disagree modestly about the relative chances of the emulation and direct-coding approaches;Read more at location 371

Note: IL DISACCORDO Edit

Our largest disagreement seems to be on the chances that a single hand-coded version will suddenly and without warning change from nearly powerless to overwhelmingly powerful; I’d put it as less than 1% and he seems to put it as over 10%.Read more at location 372

Add a note

My style is more to apply standard methods and insights to unusual topics. So I accept at face value the apparent direct-coding progress to date, and the opinions of most old AI researchersRead more at location 375

Note: LO STILE INTUITIVO Edit

putting apparently dissimilar events into relevantly similar categories. (I’ll post more on this soon.) These together suggest a single suddenly superpowerful AI is pretty unlikely.Read more at location 380

Note: LO STILE ANALOGICO Edit

Eliezer seems to instead rely on abstractions he has worked out for himself, not yet much adopted by a wider community of analysts, nor proven over a history of applications to diverse events.Read more at location 381

Note: RAZIONALISMO SPECIFICO Edit

Chapter 8 Abstraction, Not AnalogyRead more at location 505

Note: 8@@@@@@@@@@@@@@@ Edit

I’m not that happy with framing our analysis choices here as “surface analogies” versus “inside views.”Read more at location 507

Note: SURFACE VS INSIDE VIEW Edit

More useful, I think, to see this as a choice of abstractions. An abstraction (Wikipedia) neglects some details to emphasize others.Read more at location 509

Note: ASTRAZIONE Edit

For example, consider the oldest known tool, the hammer (Wikipedia). To understand how well an ordinary hammer performs its main function, we can abstract from details of shape and materials. To calculate the kinetic energy it delivers, we need only look at its length, head mass, and recoil energy percentage (given by its bending strength).Read more at location 511

Note: MARTELLO Edit

To see that it is not a good thing to throw at people, we can note it is heavy, hard, and sharp. To see that it is not a good thing to hold high in a lightning storm, we can note it is long and conducts electricity. To evaluate the cost to carry it around in a tool kit, we consider its volume and mass.Read more at location 516

Add a note

Whether something is “similar” to a hammer depends on whether it has similar relevant features.Read more at location 525

Note: SIMILITUDINE Edit

The issue is which abstractions are how useful for which purposes, not which features are “deep” vs. “surface.”Read more at location 528

Note: LA QUESTIONE Edit

The future story of the creation of designed minds must of course differ in exact details from everything that has gone before. But that does not mean that nothing before is informative about it.Read more at location 532

Note: AI E LE ANALOGIE Edit

Yes, when you struggle to identify relevant abstractions you may settle for analogizing,Read more at location 535

Add a note

Analogies are bad not because they use “surface” features, but because the abstractions they use do not offer enough relevant insight for the purpose at hand.Read more at location 536

Note: IL BRUTTO DELL ANALOGIA Edit

I claim academic studies of innovation and economic growth offer relevant abstractions for understanding the future creation of machine minds,Read more at location 538

Note: AI. ANALOGIE CON INNOVAZIONE E CRESCITA ECONOMICA Edit

previous major transitions, such as humans, farming, and industry, are relevantly similar.Read more at location 539

Note: ANALOGIE: HOMO SAPIENS AGRICOLTURA INDUSTRIA Edit

You have previously said nothing is similar enough to this new event for analogy to be useful, so all we have is “causal modeling” (though you haven’t explained what you mean by this in this context). This post is a reply saying, no, there are more ways using abstractions; analogy and causal modeling are two particular ways to reason via abstractions, but there are many other ways.Read more at location 560

Note: MODELLO CAUSALE Edit

Everything is new to us at some point; we are always trying to make sense of new things by using the abstractions we have collected from trying to understand all the old things.Read more at location 621

Note: IL RADICALMENTE NUOVO Edit

I said the abstractions I rely on most here come from the economic growth literature