Elon Musk and the Immortal dictator

[NichG]

... I am again amazed at how hard a time you can have understanding even the abstract of a scientific paper if you are not involved in the field. I mean, I think I kind of get it but "understand" is something else. I don't think I'll attempt to read the whole thing to not embarrass myself further.
Though I would say labeling the AI superstitious from this is a bit buzzfeed-y.

Curious author, though

The sense in that it's superstitious is that one of the failure modes that limits the application of this method is that if there are sources of randomness in the environment, the inner optimization loop the agent uses to plan also optimizes over those random variables, not just ones the agent actually can control. So the agent ends up acting like it can control those random variables. In a soccer-like task, even things like the prediction error distribution from grazing impact can become this sort of random variable, so the AI ends up doing a little dance in place and if you look into its predictions it believes that this will make the soccer ball impact a wall just slightly differently and therefore careen on its own towards the goal, but of course that doesn't end up actually happening.

So in the subsequent paper, we had to switch to models that made it explicit which degrees of freedom the agent could control and only optimize over those. Which works a lot better.

[Wardog]

Oh my god, the possibility of Microsoft making an AI. Forget other people screwing with it. They'd release the Windows ME of AIs (95, Vista and 8 work for that joke as well, depending on your age) and it would enjoy several years of competition-free dominance until they'd come out with a non-crazy version.

"I have a newer version of the firmware".

[Frozen_Feet]

Part of me wonders why nobody has this concern about neurosurgeons who figure out how to make themselves smarter and bootstrap themselves into superintelligence.

By my reckoning, it has a degree of plausibility over computer intelligences. We're pretty confident that neurosurgeons already exist.

No-one is afraid of neurosurgery because in plain terms, you can't add to a living brain in any meaningfull way with modern means, and you can't improve a human's mind by much by cutting things off.

There has been some discussion about using genetics and eugenics for the samd effect, but there's both a relatively hard cap and diminishing returns there as far as current knowledge goes.

The current stare of genetic engineering is such that we could maybe prevent (or cause) certain personality disorders, chromosomal defects and other developmental disorders, provided gene therapy is applied when an individual is growing up, ideally when they're still in the womb or before. It still takes as long as always for the recipient of the treatment to grow up. Gene therapy for adults has limited effect because existing tissue can't just be swapped away.

The barrier for genetic means is the human gene pool. Shortly, no matter how you mix and match various human inheritances, you cannot get a human who would be vastly more intelligent than those already existing. The current genepool simply does not contain elements that would allow for necessary changes in neural connectivity, skeletal morphology, brain size etc. You'd have to introduce either artificial genes or genes from outside human genepool. And there you have to face the fact that there's no magic intelligence gene. For even slight improvements, you need to alter multiple variables and, again, wait for test subjects to grow up so they can be tested and analyzed.

Tl;dr: biological artificial intelligences are not any easier than electronics. To add insult to the injury, modern research of biological intelligence is dependent on electronics and electronic AI, so the chances of it outpacing electronic AI research are pretty much nill.

[Andor13]

No-one is afraid of neurosurgery because in plain terms, you can't add to a living brain in any meaningfull way with modern means, and you can't improve a human's mind by much by cutting things off.

There has been some discussion about using genetics and eugenics for the samd effect, but there's both a relatively hard cap and diminishing returns there as far as current knowledge goes.

The current stare of genetic engineering is such that we could maybe prevent (or cause) certain personality disorders, chromosomal defects and other developmental disorders, provided gene therapy is applied when an individual is growing up, ideally when they're still in the womb or before. It still takes as long as always for the recipient of the treatment to grow up. Gene therapy for adults has limited effect because existing tissue can't just be swapped away.

The barrier for genetic means is the human gene pool. Shortly, no matter how you mix and match various human inheritances, you cannot get a human who would be vastly more intelligent than those already existing. The current genepool simply does not contain elements that would allow for necessary changes in neural connectivity, skeletal morphology, brain size etc. You'd have to introduce either artificial genes or genes from outside human genepool. And there you have to face the fact that there's no magic intelligence gene. For even slight improvements, you need to alter multiple variables and, again, wait for test subjects to grow up so they can be tested and analyzed.

Tl;dr: biological artificial intelligences are not any easier than electronics. To add insult to the injury, modern research of biological intelligence is dependent on electronics and electronic AI, so the chances of it outpacing electronic AI research are pretty much nill.

An additional element is that genes are only a part of what determines intelligence. You could clone Einstein, but not wind up with a genius, depending on environmental factors. The brain is incredibly plastic, and wires itself in response (partly) to it's challenges and inputs, and of course, is heavily effected by diet as well (feed baby Einstein enough lead paint chips and you're not getting a lot of math out, probably.) The simple fact is that the human brain is the most complex system that we know of in the observable universe, and we do not have a firm handle on how it all works.

The environment (in every sense) is currently changing too fast for us to have any predictive power about how those changes will effect us, look for example at the current hulabaloo about the effects of social media and smart phones on the younger generations. Are they having an effect? Obviously. Is that effect good or bad? Both, probably. We're going to have the technical capacity for computer mediated human telepathy in the next 10-20 years, what we're going to do with it, and what effects that will have, no one knows.

[halfeye]

The barrier for genetic means is the human gene pool. Shortly, no matter how you mix and match various human inheritances, you cannot get a human who would be vastly more intelligent than those already existing.

The variance in intelligence between humans is huge, if you shifted every new human to the high end of the current spectrum, society would be dramatically different.

[brian 333]

Isaac Asimov wrote about this in the 1950s. (Franchise, Univac, I Robot, etc.) Musk is just the latest to repeat the idea.

He clarified the concept in his 1980s Prelude To Foundation stories which linked his Galactic Empire stories and his Robot stories. R. Daneel Olivaw made a great Emperor.

Oh, and Asimov had mind-reading AI, so add that to your prognostications.

[Frozen_Feet]

The variance in intelligence between humans is huge, if you shifted every new human to the high end of the current spectrum, society would be dramatically different.

You didn't really stop to think of the logistics of giving designer babies to every breeding couple in existence, did you?

Also, any parent worried about their smarter and better offspring replacing themselves is kinda dim.

[halfeye]

You didn't really stop to think of the logistics of giving designer babies to every breeding couple in existence, did you?

It wasn't in the question/quibble I was answering, so no, why would I? Being very intelligent is a thing, it's as much a problem as it is a power but it is a difference. There is a possibility that low IQ babies are the result of some deficiency, if that could be fixed, then everyone would be brighter. It is a difference, and it would make a differece to society, whether that difference would be bad, who can say?

Also, any parent worried about their smarter and better offspring replacing themselves is kinda dim.

Yeah, but half of the population is below the median in any measure.