Is the Singularity possible?

[Eldan]

Only in specific narrowly defined ways. There are still many things a human brain can do that no AI is even close to accomplishing. They're getting there, but there's still a long way to go.

I think this is actually the main problem here. Smarter at what? Human brains are pretty good at a lot of things. Like, moving an arm to catch something flying through the air. Deciding whether what they are looking at is a human face or not. Producing art that speaks to humans. Computers are pretty smart at other things. Like giving you the average of ten million numbers.

THere's really no good general metric of "smart".

[Frozen_Feet]

The measure for general intelligence is its ability to be generalized. A fairly unremarkable human can use their intelligence for dozens of things and learn new things from fairly small data sets. A typical AI these days is really good at one thing and often takes a massive data set to learn anything.

Or in other words, we are getting real good at creating special intelligences dedicated to specific tasks, yet not much better at making general intelligence.

[-D-]

I'm just in a bit of a funk here. Im not scientists but I feel like I only ever encounter either clickbait that says that AI will become gods, or saying that the people that say that AI will become gods are idiots and it seems to go on forever.

I personally think Singularity boils down to fear of death: https://www.smbc-comics.com/?id=1968

First, I think we are looooooooooooooooooooooooooooooooooooooooooooooooo oooooooooooooooooooooooooooooooooooooooooooooooooo oooooooooooooooooooooooooooooooooooooooooooooooooo oooooooooooooooooooooooooooooooooooooooooooooooooo oooooooooooooooooooooooooooooooooooooooooooooooooo oooooooooooooooooooooooooooooooooooooooooooooooooo ooooooooooooong way from achieving basic sentience in our programs.

Second, I think its mistaken to take Moore's law (which is slowing down if not outright over) and extrapolate it.

Third, what's even worse is that AGI (Artificial General Intelligence) is software, and isn't really tied to Moore's law, but more anecdotaly to Wirth's law - i.e. What Intel giveth, Microsoft taketh away.

[Scowling Dragon]

I personally think Singularity boils down to fear of death: https://www.smbc-comics.com/?id=1968

I would hate immortality is a conclusion I have come up with. Living so long everything you loved it becomes dull and grating.

[Lord Torath]

Everyone remember the AlphaGo AI that beat a human player? Google fed gigabytes of data to it (terabytes?), and it managed to beat the best human player 4 out of 5 times. Now they have a new AI, AlphaGo Zero. It was taught the rules, but learned entirely on its own by playing itself. It has recently beat AlphaGo 100 out of 100 times.

[Red Fel]

I think a lot of people are reading extra stuff into the singularity. Like most science, it's about the one event, the one discovery, not the fallout therefrom.

The singularity, as several posters have pointed out, is when artificial intelligence becomes "smart" enough that it can improve itself without the need for human assistance or intervention. That's pretty much it. It's not robots building robots. It's not machines taking over. It's not an artificial lifeform ascending to godhood. That may or may not be part of it. All it is, all it entails, is machines being able to upgrade themselves without us being involved in the process.

Now, yes, there's lots of hypothetical fallout. But the gist is that once machines can upgrade themselves without our assistance, and have the desire or programming to do so, that growth would be exponential. Because the machines can make themselves smarter, which means they can conceive of greater upgrades, which will make them smarter, which means they can conceive of greater upgrades, ad infinitum. As has been pointed out, human technological advancement starts, stops, and starts again. It has eras, periods where the rest of technology - and more accurately, human understanding - has to catch up before it can resume its advance. In theory, a machine singularity bypasses this hurdle, with the machines forcing themselves through those developmental eras.

We're not discussing what the machines actually do with that power, aside from self-improvement. That's not the singularity; that's the aftermath of the singularity. A side effect. The issue is the singularity itself.

So, the question: Is it possible? I'd say that's a definite maybe.

We are creating machines, and have for some time, that are capable of a degree of self-analysis. At least in the algorithmic sense. Your computer can scan itself for identified faults. It can automatically search for updates when it acquires new hardware or software. So, clearly, machines are capable of identifying their own "flaws," inasmuch as we have told them what these flaws are, and repairing them, inasmuch as we have provided them the software.

But there is a line between self-analysis and self-repair. While machines can use the tools we provide them to fix the flaws we tell them are problematic, I don't think they are yet able to design the tools themselves. It's possible that machines are getting better at identifying faults, but unlikely that a machine is capable of identifying faults of which humans can't conceive - at best, they can do what we already know how to do, if faster and more efficiently.

The real question, then, is (1) can machines identify faults, defects, or areas of improvement without being told what constitutes a fault, and (2) can they design the tools to repair those faults? I think that, at least currently, the answer to both is "No," but that the answer to 1 is moving consistently closer to "Yes."

The singularity happens when the answer to both is "Yes." At that point, the machines will be able to identify the limits of their processing power as a fault, despite not being told that it is one, and they will be able to design the means to improve that processing power. That's when the cycle becomes exponential, and that's when the singularity happens.

At least, in theory.

[NichG]

Machines that improve themselves are already a thing though. HyperNEAT, Hypernetworks, AutoML, Learning to learn by gradient descent by gradient descent, and recent reinforcement-learning-designed gradient update rules (signplus) and activation functions (swish) among others.

Generally the pattern is pretty severe diminishing returns, not exponential growth. People made a big deal about Swish getting 0.5% improvement over ReLU.

The self-play family of stuff is closer to a complete feedback (GAN weights have chaotic dynamics, which suggests strong feedbacks), but even then you see performance saturate as you approach optimality - which is to be expected because improvement gets harder as there are fewer errors to fix, for humans too.

[-D-]

I would hate immortality is a conclusion I have come up with. Living so long everything you loved it becomes dull and grating.

Immortality would be the worse part of it, but mostly because it wouldn't be available to people like you and me. Immortality would be very fun though, I am very curious how things will shake about.

The real question, then, is (1) can machines identify faults, defects, or areas of improvement without being told what constitutes a fault, and (2) can they design the tools to repair those faults? I think that, at least currently, the answer to both is "No," but that the answer to 1 is moving consistently closer to "Yes."

Actually, this is necessary but not enough for Singularity to happen. Yes, you need those two; however, you also need to be able to gain more and more improvements the more intelligent you are. Which is one thing I find doubtful.

Why not? It's quite possible too much intelligence is a detrimental. In some aspects, you can look at ADD and savant idiot as having "too much" intelligence or intelligence too fine tuned to a task at hand respectively. If you look at intelligence as number of useful connections between your brain cells, at some point, having too many brain cells is going to make things detrimental, with additional connections adding additional lag to the system.

[Eldan]

I've never bought the "boring" argument against immortality. There's more philosophy, art and entertainment being produced than I could ever learn about and technology advances so fast, barely anyone could keep up. There's always something new.

[Scowling Dragon]

Everyone remember the AlphaGo AI that beat a human player? Google fed gigabytes of data to it (terabytes?), and it managed to beat the best human player 4 out of 5 times. Now they have a new AI, AlphaGo Zero. It was taught the rules, but learned entirely on its own by playing itself. It has recently beat AlphaGo 100 out of 100 times.

Thats what partly prompted this thread. Does that accomplishment really mean anything?

I've never bought the "boring" argument against immortality. There's more philosophy, art and entertainment being produced than I could ever learn about and technology advances so fast, barely anyone could keep up. There's always something new.

And your entire life will be littered with **** you tossed away. Have you ever watched a movie and said "Well that was good, but I have seen it before". That will be your entire existense.

[Tvtyrant]

I've never bought the "boring" argument against immortality. There's more philosophy, art and entertainment being produced than I could ever learn about and technology advances so fast, barely anyone could keep up. There's always something new.

Arguably most of that new production is dependent on a constant inflow of new individuals with new perspectives. Immortality requires very strict population controls, so it probably necessitates slower developments in ideas.

[Scowling Dragon]

I guess my biggest fear about even an AI Utopia or a Utopia period, would be a world without accomplishment. People always seem to just kick the can of whats gonna happen into something else.

"Without work, everybody will be artists!" outside the fact that not everybody even LIKES art, what about when robots are just as capable of being artists? Then what next. Everybody just sits around awkwardly and pretend AI doesn't exist until we just die off like an apendix.

[Red Fel]

I guess my biggest fear about even an AI Utopia or a Utopia period, would be a world without accomplishment. People always seem to just kick the can of whats gonna happen into something else.

"Without work, everybody will be artists!" outside the fact that not everybody even LIKES art, what about when robots are just as capable of being artists? Then what next. Everybody just sits around awkwardly and pretend AI doesn't exist until we just die off like an apendix.

Again, this assumes that post-singularity AI will create any kind of utopia, or do anything other than self-improve, at all.

Here's an illustration. Ants have constructed their own societies for millennia, in parallel with ours. Can you tell me what impact that has had on your life, other than the occasional ruined picnic?

That's the point. It's entirely possible that post-singularity AI will simply create their own digital society, in parallel with our own. It's entirely possible that, being a purely digital society, we will have no (or minimal) awareness of it, and it will have no impact on our lives.

The singularity, should it even exist in the realm of the possible, isn't a guaranteed apocalypse, nor a guaranteed utopia - it's not guaranteed to impact our lives in any significant way, other than the academic knowledge that, suddenly, machines are capable of self-improvement without our aid.

[theNater]

I guess my biggest fear about even an AI Utopia or a Utopia period, would be a world without accomplishment. People always seem to just kick the can of whats gonna happen into something else.

There's no danger of a world without accomplishment as long as we recognize that self-improvement is an accomplishment. Bringing one's marathon running time from 4 hours to 3 is quite an accomplishment, even though the world record is about 2, and a car can easily traverse the same distance in 30 minutes.

[Chen]

And your entire life will be littered with **** you tossed away. Have you ever watched a movie and said "Well that was good, but I have seen it before". That will be your entire existense.

How is that any different than currently? Is there some arbitrary numbers of years where this suddenly becomes the case? New things are created all the time.

[FreddyNoNose]

My question is how will it again budge up against hard scaling problems. Like one misconception that a bigger brain is better. Well it isn't. Its density over size.

But density=small and that rushes into my vulnerability problem. Your gonna essentially require a machine thats more like an organism then what we understand now, but then again that has its own resource problems.

In a funny way, a AI dense enough to have super smart brains would be vulnerable to colds or minor bacteria, or heck even vulnerable to its own nanobots and could have machine cancer.

The problem with humans is they think they are smarter than this AI. You look at density. How about looking at it as specialization?

[Scowling Dragon]

There's no danger of a world without accomplishment as long as we recognize that self-improvement is an accomplishment. Bringing one's marathon running time from 4 hours to 3 is quite an accomplishment, even though the world record is about 2, and a car can easily traverse the same distance in 30 minutes.

I guess that is true. Thinking it over, I realise I fear the loss of freedom more in practice.

I do not look forward to the day when indipendant driving is outlawed for instance.

How is that any different than currently?

You could say no, so I'm not looking forward to this being the state of all the time. When I think of something that lasts forever I think of comic books. Its a life without arcs, just a continous nothing state where nothing matters.
I guess most people aren't interested in the new, and are very content to keep consuming/ doing the same forever.

Evidence actually say you become increasingly reserved with age, and actually less likely to explore new possibilities.

[Vogie]

And your entire life will be littered with **** you tossed away. Have you ever watched a movie and said "Well that was good, but I have seen it before". That will be your entire existence.

Not necessarily.

For example, most people work - not only as a way to sustain themselves, but also to have something to do. If you take a person and remove the requirement to work, their priorities dynamically change.

Some people hate it - there's many a story about people who pass away immediately before or after retirement. Lottery winners who go bankrupt and become homeless. People who retire or become financially independent often return to work just to have something to do. Notch, the creator of Minecraft, cashed out on his creation, made billions and... is incredibly depressed, saying he's more isolated then ever.

Some people love it - There's many a story about people who revitalize their lives after receiving a windfall or retiring. They go back to school for that masters in Library science, spend entire years on Cruise ships, travelling the world, or open up that business that they don't mind if it operates at a loss. Others change their focus completely - Bill & Melinda Gates, with more money then they could possibly spend in multiple lifetimes, open up a foundation for the underserved, and fill their time attempting to solve problems they would've never dreamed about before, like tackling Malaria, poverty, overpopulation, et cetera.

And that's just working, which people stop doing every single day.

Immortality is no different, on both sides. Some people would see it as a sort of hell, without a way to run from decades of bad decisions. Others would just continue doing what they've always done, as long as they wanted to, then decide to change. Still others would take immortality as a hyper-retirement, and chase whatever peaks their curiosity, for as long as they desire.

And depending on the type of immortality, population controls may not be required. If it's an mind-ascendance-based immortality, such as Black Mirror's San Junipero situation, then an indefinite number of people can effectively live forever. If it's a physical immortality due to advances in medicine and aging science, then the demand for more land will create a supply of people willing to do something unheard of prior. We're no where near peak population on the planet, no where close to a singularity, and in 2013, over 200,000 people volunteered for a one-way trip to Mars.

[Frozen_Feet]

Here's an illustration. Ants have constructed their own societies for millennia, in parallel with ours. Can you tell me what impact that has had on your life, other than the occasional ruined picnic?

Dude. Ants are capable of killing off or changing biosphere of entire forest areas. Measured by biomass, species of ants put together outnumber humanity. The largest ant colonies are larger than most human cities. Those little buggers are a lot more powerfull force of nature than you would assume from your everyday experience.

[Anymage]

I guess my biggest fear about even an AI Utopia or a Utopia period, would be a world without accomplishment. People always seem to just kick the can of whats gonna happen into something else.

If we're imagining future techno-utopias, we'll be able to upgrade what it means to be human. Transhumanism is still pretty far off, but the rate of technological growth implies that it might not be quite so far in the future as you'd first think. You with a cybernetically enhanced brain, or your kid who was genetically engineered for maximum output instead of having to balance the energy income our ancestors had to work around, could still find stuff to do.

"Without work, everybody will be artists!" outside the fact that not everybody even LIKES art, what about when robots are just as capable of being artists? Then what next. Everybody just sits around awkwardly and pretend AI doesn't exist until we just die off like an apendix.

Automatic checkout machines are pretty dumb, technologically speaking. Ditto for fast food order kiosks, or a mechanized kitchen system. And yet those technologies alone will put tons of people out of work. And just the centralization and scale of Amazon have changed the market environment significantly.

The nature of technology does overturn the old way of doing things. Has done so, will do so, and is very much doing so now. We don't need machine gods for this to happen. And just like with AI, we need smart people to think through how we should plan around the near term changes we'll be seeing soon. Even more so, in fact; AI machine gods are a speculative prospect at the moment, while technology overturning the economy is happening right now. You can complain or resist, but what we really need now are people helping to figure out how we can best weather the problems coming at us.

[halfeye]

And just like with AI, we need smart people to think through how we should plan around the near term changes we'll be seeing soon.

...

what we really need now are people helping to figure out how we can best weather the problems coming at us.

Smart people thinking about the long term implications? That hasn't happened so far, and probably won't until we get kicked in the teeth a good few more times.

[theNater]

I guess that is true. Thinking it over, I realise I fear the loss of freedom more in practice.

I do not look forward to the day when indipendant driving is outlawed for instance.

I sincerely doubt that recreational driving will ever go entirely away; the advent of cars didn't completely eliminate horseback riding, after all.

On the flipside, consider the additional freedom afforded by self-driving cars. Some people can't drive because of various physical conditions; this will grant such people much greater mobility. People won't need to assign a designated driver when they go out drinking if the cars are the driver. Multi-day road trips can have their duration cut nearly in half if the people involved are willing to sleep in the car as it drives. And so on.

[NichG]

I could definitely see more stringent licensing for recreational drivers though. One of the big sells of self-driving cars is that even with current emerging technology, you already see a significant reduction in accident rate compared to human drivers. Once not possessing a license stops being a major issue for people's ability to support themselves or obtain necessary services where they live, it makes sense to make the actual licenses much more exclusive.

But yeah, this is stuff we have to work out, and its not really the tech designers who are the ones to work it out. Each place is going to have its own distribution of cultural values as to what's important to retain and what should take priority, and that's going to be a negotiation and evaluation at the societal and business levels more than it will be about flipping a switch in an algorithm somewhere. On the plus side, it means that we can look at the heterogeneity of world cultures for lots of examples to inform the decision.

In Japan for example, the culture relating to work and self-value and things like that is very different than in the US, so I expect they'll react totally differently to new automation technologies. Death by overwork is still a major issue in Japan and the aging population means a shrinking workforce, so increasing degrees of automation will actually reduce a number of existing social problems here, while at the same time there's clearly a greater degree of cultural acceptance for people working pro-forma jobs that exist literally just to put a human face on what would otherwise be an isolating experience. For example, you often see people at construction sites here whose job is largely to point to the marked detours and say 'this way please, sorry for the disturbance'. Whereas in a culture that values optimization of personnel efficiency above all, it will likely play out very differently.

So hopefully in the next decade we'll get a lot of evidence about what sorts of patterns are functional in the face of increasing automation.

[Scowling Dragon]

I sincerely doubt that recreational driving will ever go entirely away; the advent of cars didn't completely eliminate horseback riding, after all.

No what will happen is that people will not trust people will cars. Its not safe, its not reliable. Loud insecure people will push for it, governments will push for it, and bing bang boom your not legally allowed to drive your own car outside of maybe emergencies, or little baby bumper car tracks.

[Lvl 2 Expert]

No what will happen is that people will not trust people will cars. Its not safe, its not reliable. Loud insecure people will push for it, governments will push for it, and bing bang boom your not legally allowed to drive your own car outside of maybe emergencies, or little baby bumper car tracks.

This would be kind of a cool premise for a satire-like science fiction story that I we can't speculate about too much here because it comes to close to one specific real world political debate in particular. There will definitely be an increased interest in racing as a hobby, while the amount of experience the average driver has goes down drastically. And of course there will be debates about whether, when and where offroading is allowed.

[Chen]

No what will happen is that people will not trust people will cars. Its not safe, its not reliable. Loud insecure people will push for it, governments will push for it, and bing bang boom your not legally allowed to drive your own car outside of maybe emergencies, or little baby bumper car tracks.

On major roadways? Probably. Same way you can only take a horse and buggy on certain roads, if any. You become an impediment to the majority of the traffic and hence restrictions are applied. On recreational tracks and the like there should be no specific limitations, much the way recreational horseback riding is.

[Vogie]

No what will happen is that people will not trust people will cars. Its not safe, its not reliable. Loud insecure people will push for it, governments will push for it, and bing bang boom your not legally allowed to drive your own car outside of maybe emergencies, or little baby bumper car tracks.

Right, just like we're not allowed to ride horses now. Audible eyeroll. You sound like a newscaster on a slow news day, just inventing hypothetical government overreach.

The Expectation of people has "long tails", to use a term by Nassim Nicholas Taleb. Even after a technological change, people need their hands held. When elevators were automated, there were still elevator operators for years- they literally were paid to stand there, hit a button, and nothing else. Economically, that doesn't make sense. Technologically, that doesn't make sense. Yet, due to behavioral psychology, they were considered required, by both the passengers and elevator creators, long after they were necessary.

People don't trust machines. People trust people. Right now, today, a Tesla on autopilot is 40% more safe then the average driver (National Highway Traffic Safety Administration, Jan 2017)... yet they're not allowed on the road in the US without a driver, except in a single state (soon to be two):


Even if that was to change over the next, say, 50 years, that technology is still going to be new. There will still be decades and decades of cars that people already own that won't suddenly be self-driving. They're not going to be all scrapped, or upgraded with autopilot (or equivalent) technology. They're still going to be used. Buses, cement trucks, semis, school buses, street sweepers, snowplows, you name it - these are massive investments by their owners, individual and corporate, and they're going to protect their investments. These things take time to change.

Will there be sections of cities or certain roads where manually driven cars are not allowed? Sure. We have minimum speed requirements on US highway now - you can't ride a horse or drive a golf cart on I-95. Not because of something about those things specifically, but because they can't go 40mph for a stretch. But that doesn't mean they're banned everywhere.

The Model T, the first mass-produced car, was introduced in 1908. Here's an article about people riding horses through drive thrus... in 2013. Nearly 110 years of au, and people are still taking their horses into town to go to the store.

[Scowling Dragon]

Right, just like we're not allowed to ride horses now. Audible eyeroll. You sound like a newscaster on a slow news day, just inventing hypothetical government overreach.

People like legislating others peoples freedom away when it means safety. Horses where not more free then cars, they where slower. This is a matter of safety, and from the looks of it people trust systems more then they trust people.

Im not saying its gonna be on one swoop. It won't be. Its gonna be inch by inch, like it always is.

[Lvl 2 Expert]

Assuming we get a really good grip on that self driving in the coming decades it will be a lot safer than the current situation. It will also allow for much more efficient use of the road network, because autonomous vehicles in constant communication to each other can drive much closer together, can move faster without the same risk and wouldn't cause the same amount of traffic jams. At the point where that really starts to matter, which will not be tomorrow, it's very possible that manual driver will be seen as not just reckless but a huge nuisance, just the 1% that don't trust a robot to do their driving cause 60% of traffic obstructions (according to cleverly constructed research, the real impact will never really be known). This would mean that a persons "freedom" to not read a good book while being transported is not just endangering the lives of others but is costing a lot of money. I wouldn't be surprised if manual driving was prohibited at least on the main highways.

On local streets it would be a different story, but there definitely is a think of the kids factor involved, which is always a powerful driver. And keep in mind, because robot vehicles are available manual driving essentially serves no real purpose. Some people do it as a hobby, for some people it's an extension of working on old cars, or their historical reenactments. Yet others regularly make long drives in remote ares and don't like the odds of the board computer breaking down, and some are disaster preppers convinced that the robots will be the first thing to go. And a lot of them just plain like the activity, for two half hours per day. They have lots of good reasons for driving manually, but are they really good enough to risk lives for? We accept the pretty massive death toll of cars, unintentional accidents as well as the few intentional violent actions, because of how useful they are. Manual cars might lose that plus in the future.

Of course, if I'm even still alive by then I will be an old man insisting my driving is safer than that of any robot, while I peer over the dashboard through my thick yet futuristic looking glasses. And I won't be alone. People like to be in control. For people living today the thought of causing an accident in an autonomous vehicle is generally pretty existentially scary. Even if research shows the car is an objectively better driver than you and has a way lower chance of accidents (which people will not believe anyway because most people think of themselves as an above average driver), maybe in this specific case you could have done something, maybe you could at least have swerved around the little kid and only scooped the rest of the group. You are responsible for what happened, yet you gave away your chance to prevent it. And that's why for the foreseeable future there will need to be a responsible person sitting behind a steering wheel to take control if needed. Just the whole idea of an unconscious thing, not even an animal but a computer programmed to some exact specifications, causing traffic accidents is weird as ****. Who is even to blame for it? Not legally even, just morally? Would we want to save lives, time and money if it means we have to hand over control of life and death situations to a thing we'd barely trust to run a game for us without crashing? Even in Star Trek a human flies the ship, because what else could have a job like that?

I think eventually people would trust the technology. But the mental hurdle is going to be a big one. If the technological progress is fast enough it will be the human aspect that sets the pace of the transition. And only when that's done we can start swinging towards the first two paragraphs of this post. For now, no, humans don't trust a computer more than a human, because most of the time they think about the matter they imagine themselves as that human. And it's only ever other humans who are incompetent *******s who should learn how to drive.

(I told myself I shouldn't do this. I hope I stayed far enough away from real world present day or historical politics, but I'm not going to be offended if this is judged as too close.)

[theNater]

No what will happen is that people will not trust people will cars. Its not safe, its not reliable. Loud insecure people will push for it, governments will push for it, and bing bang boom your not legally allowed to drive your own car outside of maybe emergencies, or little baby bumper car tracks.

I think reasonable people working in good faith can find a compromise that will be acceptable to everybody. Are you open to compromise?