Superintelligence: Paths, Dangers, Strategies

I enjoyed parts of this book, but overall found it quite a chore to get through. Just a few years on, some things have changed so much that parts of the book feel dated already. I enjoyed catalog of the problems, approaches and dangers, but found some of the discussion of them pedantic.

Just a few annotations:

“A big breakthrough in artificial intelligence. It seems somewhat likely that it will happen sometime in this century, but we don't know for sure”.

-I don't think so, but I truly hope to see that with my eyes.

“The idea of a coming technological singularity has been popularized, starting with Vernor Vinge's seminal essay and continuing with the writings of Ray Kurzweil and others. The term ‘Singularity,' however, has been used confusedly in many disparate senses and has accreted an unholy aura of techno-utopian connotations”.

-Agree.

“Machines matching humans in general intelligence have been expected since the invention of computers in the 1940s”

-Add another century to it.

“The fact that the best performance at one time is attained through a complicated mechanism does not mean that no simple mechanism could do the job as well or better. It might simply be that nobody has yet found the simpler alternative”.

-That's my bet for developing a SI.

“How far are we currently from achieving a human whole brain emulation?. One recent assesment presented a technical roadmap and concluded that the prerequisite capabilities might be available around mid-century, though with a large uncertainty level”.

- I also doubt a WBE in the near future. Probably in 2120?.

“Final goal: ‘make us smile'.
Perverse instantiation: ‘paralyze human facial musculatures into constant beaming smiles”.

- I laughed on this because it makes sense.

“...These observations make it plausible that any type of entity that developed a much greater than human level intelligence would be potentially extremely powerful. Such entities could accumulate content much faster than us and invent new technologies on a much shorter timescale. They could also use their intelligence to strategize more effectively than we can”.

- I liked this book a lot specially because of the Realistic Pessimism about an SI. I consider there must be a real concern to think about that level of intelligence.

This is more or less the technical tour de force of artificial intelligence books currently out there. If you're looking for a light introduction to the concepts of artificial intelligence, this is not the book for you.

Nick Bostrom goes in deep into the hurdles, risks and hazards of artificial intelligence. While I suspect there are academic books that go even deeper into the math and algorithms covered, Bostrom doesn't shy away from that discussion. It feels exhaustive. It feels like a technical textbook in many ways. The material is very dense. In addition to technical discussion of approaches to achieving AI and the challenges involved with them, there is deep discussion of the risks and even the ethics of AI. In my opinion, these were the most interesting parts.

Having read quite a few other books on the subject, I felt familiar with most of the terms used, as well as many of the major concepts. Even with this, this was a much more technical discussion than anything I'd read previously. With this in mind, I'll say I didn't really “enjoy” this book. But I do feel much better educated. I'm not sure how to account for this in “my rating.” I'm going to give it a 4 because this feels like the definitive discussion of the topic.

Noiosissimo e troppo teorico. 4

Super interesting.

Tato kniha není pro někoho, kdo už se zajímá o umělou inteligenci delší dobu. Je to spíš pro lidi, kteří zrovna narazili na současný trend AI a chtějí se dozvědět více o tom, kdy se pravděpodobně dočkáme superinteligence, jak to bude s prací v budoucnosti a co hrozí lidstvu, když si nebudeme při vývoji AI dávat pozor.
Zajímavé je, že se zvýší hodnota lidské práce v oblasti umění, protože se stane vzácnějším zbožím a boháči budou chtít (vlastně celou dobu chtějí) vlastnit něco, co ostatní mít nemohou a nedokáže strojově nedokáže vyrobit žádný stroj.

You need to keep up to read this book. It's a very serious analysis of the AI world and it's future possibilities. A definite eye opener!

Should have been 1/5th the length. Reads like someone spouting hypothetical scenarios off the top of their head. The only reason it gets a 2 stars is because it isn't “The Circle” by Dave Eggers.

I'd give it 4 stars because the topic is fascinating, but the book is quite verbose and some chapters hard to get through. Also, don't expect any definite answers to the questions posed. But I guess that's the nature of imagining the future, especially on a thing as potentially important as AI.

If you're familiar with x-risk, there isn't a lot for you in this book, except perhaps a better understanding of just how risky x-risk can be. For the uninitiated, however, I would wholeheartedly recommend this book to you – it briefly chronicles what likely may be humanity's most important (and last, one way or another) achievement, and how terrible it will be if we get it wrong.

Here are the particular passages I underlined while reading through Superintelligence:
http://sandymaguire.me/books/nick-bostrom-superintelligence.html