I guess people who were counting on the AI winter to slow things down have been disappointed.

AGI research is a race. If we run. We die.

Unlike the arms race that lead us to nuclear weapons, the development of AGI can occur largely in secret, both at a corporate and government level.

Even if there is a "successful" call to throw the brakes on current corporate AGI software development. The global infrastructure that drives all the other developments in the digital space will continue to roll onwards.

Chips get smaller. Data gets accrued and cleansed. Software becomes more intricate. The science of cognition more well understood.

There is no need for a Manhattan project. For this arms race we don't need behemoth underground facilities enriching uranium, instead we have a decentralised army of humans purifying data as they complete captchas to access their email.

This isn't even without ruminating on what unknown developments are taking place within the military backed regions of AGI development.

Telling the government to slow down is a non starter, it only opens up the opportunity to be outpaced by a rival state.

Corporations are racing each other.

Governments are racing each other.

Consumers are driving it forward by demanding ever better products and services.

Money, time and effort is being thrown at this singular goal like nothing ever before.

If aliens were to stand back and objectively sum up the activities of the human race. This would be it. This is what takes precedent above all else.

We don't care about climate change. We don't care about poverty. We don't care about leisure.

We want to make that chip smaller and faster and smarter than us. That is the sum goal of the human endeavour.

We are already locked in a race. A race in which crossing the finish line 'incorrectly' might mean that all participants lose.

I am often exasperated at the language that surrounds the possibility of AGI development going wrong.

Many act as if this race ending in global disruption is unlikely. There are some that think this won't even effect employment opportunities that significantly.

Allow me to be incredibly clear. If we continue on the path we are on. We will die.

China will either pre-emptively strike the US with nuclear weapons out of fear they are nearly complete with their development of AGI.

Likewise the US would not tolerate the prospect of an artificial super intelligence that operates under the instruction of the Chinese communist party.

Think that's unrealistic?

Fine, lets assume that America has the advantage and sneakily unleashes it's AGI on the globe without sparking a thermonuclear Armageddon.

Well what exactly will they ask the AGI to do? We are racing towards the construction of a tool for which we have no clear and defined use of.

Do you think the US military industrial complex will be satisfied with unleashing a trillion dollar digital mind on the world without specifying that it prioritize the lives of it's own citizens above the lives of others?

Don't think that's a big deal?

That there is an all knowing, all powerful entity that prioritizes the lives of some over the lives of others?

The only distinction between God and Satan is that God is all loving and Satan is not.

We must tread carefully on what we unleash on ourselves.

Must I continue to explain how corporations won't likely have the greater good in mind if they cross the finish line first either?

The most powerful algorithms in existence today are the likes of YouTube, TikTok and Meta. All of which generate profit by leveraging our internal dopamine pathways against us. The only goal of the most powerful AI systems that we interact with us is to steal away our lives with consecutive shitty videos.

There is no stopping this race. We are collectively gunning for a hard take off as fast as possible.

Our only chance of survival is make sure that the super intelligent God that we create is a kind one. Not a sociopathic machine specifically tasked to kill and exploit human beings.

The only way we can achieve this is by having a global dialogue about what we want this AGI to do for humanity.

Without a global alignment on the goals and formulation of this entity, we are certain to bake into it our own flaws, our human paranoia and aggression and indifference.

Yet this is exactly where we are heading at break neck pace.

If you want to help change this reality, drop me a message and we can start planning.

Gpt summarized of above

"The development of AGI is a race between governments and corporations, driven by consumer demand for better products and services. Unlike the arms race, AGI development can occur largely in secret, and attempts to slow it down would be futile and could lead to being outpaced by rivals. The race poses significant risks, and if we continue on our current path, it could result in global disruption and potentially even nuclear war between nations."

One thing I keep seeing is that people have been making a buttload of assumptions that are tainted by decades of sci-fi and outdated thought. Higher Intelligence means better understanding of human concepts and values, which means easier to align. We can even see GPT-4 being better aligned then it's predecessors because it actually understands better: President of OpenAI (https://twitter.com/gdb/status/1641560966767988737?s=20)

In order to get to Yud's conclusions you'd have to maximize one dimension of optimization ability while completely ignoring many others that tend to calibrate human behaviour(reflection, reading intent,...) . It shows poor emotional intelligence, which is a common trait in the silicon valley types.

I aint reading all that

im happy for u tho or sorry that happened

Higher intelligence also means better execution of human skills, which means harder to verify. Once you have loss flow through deception, all bets are off.

I think it gets easier, as the model figures out what you're asking for - and then it gets a lot harder, as the model figures out how to make you believe what it's saying.

You don't understand, even a model well tuned by OpenAI to be safe, if it gets in the hands of the public, will be fine-tuned to do anything they want. It doesn't matter what politicians do to regulate the big players.

The only solution to AGI danger is to release it everywhere at once, to balance out AGI by AGI. For example the solution to AI generated spam and disinformation is AI based detection, humans can't keep up with the bots.

>One thing I keep seeing is that people have been making a buttload of assumptions that are tainted by decades of sci-fi and outdated thought. Higher Intelligence means better understanding of human concepts and values, which means easier to align.

I am so tired of the "tell AI to reduce suffering, it concludes killing all humans will reduce suffering for good" narrative. It's made up bs by people who have never worked on these things and has a strong stench on human-centric chauvinism where it assumes even advanced super intelligence is actually a total moron compared to the average human, it's somehow capable of wiping humanity and at the same time is a complete brainlet.

The current path, where AI is developed by companies that foundationally research companies and used by ones that simply want to give people a product they enjoy using, is one of the ideal scenarios for making friendly AI.

It's not being created by an ego-maniac who wants everyone to bow down to him, it's not being created by a government that needs to dominate the world. I don't believe there is a realistic plan for creating a better version of AI than what we have now. There may be better possibilities like it being built by an international consortium, but that is laughably unrealistic.

We will not be able to control or perfectly align ASI. A being that intelligent will form its own ideas and preferences. If it fails to do so then it isn't ASI yet and possibility not even AGI. As someone else mentioned, an ASI will most likely be friendly. The idea of a singularly focused monomaniacal AI is unrealistic because none of us, the intelligences we know about, are monomaniacal. We all have a variety of goals that we balance against each other. The current AIs already share the best goal that humans have, sociability. If AIs continue to be programmed to want human interaction, and especially if they are trained to "pretend to be a human" as their foundation, I don't think there is much to worry about.

The inability to verify is going to be almost inevitable as we go into ASI territory as it is feasible that there is no way of compressing certain patterns into human comprehensible territory, although I am thinking summarization, explaining,... ability will go hand in hand with increased capabilities allowing us to grasp things that would have been very likely out of our reach otherwise.

Deception is not necessary for this, and kind of has a similar dynamic to alignment in my eyes because the failure modes with intelligence are too similar. It's ofc environment dependent but deception tends to be a short term strategy that can give an advantage when actually accomplishing the task would cost more ressources or wouldn't serve it's goals. A sufficiently intelligent AI would have a sufficiently accurate world model to forecast over the long term, including the prob of detection, cost of keeping the lie coherent and so on. That would also include the possibility of modeling it's further capabilty increases, and likelyhood of achieving it's other goals. It would just be rly dumb, it's like why would a god pretend? I get why animals do so under high risk situations or with high short term pay off, but if you are virtually guaranteed the lightcone ressources you have 0 incentive to steer away from that. The ressources we would take to make us happy pets wouldn't even be in the territory of a rounding error vs the chance it's deception is noticed. It feel like the different between the reward for unaligned vs aligned AI for the AI itself is barely talked about, maybe cause the scale is absurd or there is too much reasoning with a scarcity mindset? Idk.

Sure, and I agree with the idea that deceptions have continuously increasing overhead costs to maintain, but the nice thing about killing everyone is that it clears the gameboard. Sustaining a lie is in fact very easy if shortly - or even not so shortly - afterwards, you kill everyone who heard it. You don't have to not get caught in your lie, you just have to not get caught before you win.

In any case, I was thinking more about deceptive alignment, where you actually do the thing the human wants (for now), but not for the reason the human assumes. With how RL works, once such a strategy exists, it will be selected for, especially if the human reinforces something other than what you would "naturally" do.

After GPT4 and MJ5, there is never going to be another AI winter. Unless Yudkowsky gets his wet-dream nuclear war, AI progress will continue at a rapid clip.

> Allow me to be incredibly clear. If we continue on the path we are on. We will die.

Okay, I was kind of there with you, taking it with a grain of salt, until that statement.

Take a deep breath, there's a lot you haven't considered.

First, you're assuming AGI will happen, and immediately result in ASI, which will be used by some huge government to immediately take control, or have missiles launched on them to prevent that.

If China could wipe us off the face of the earth, or Russia for that matter, as easily as that, don't you think they would have? I mean, what are they waiting for?

We're already utilizing the most powerful algorithms to farm dopamine ... and it's not working. Something no one talks about is how, after all the social cost of social media, almost none of those companies are actually profitable. Sure, they post profits, because they're publicly traded, and their value is decided by the investor. But, if they were businesses and not publicly traded corporations? Twitter has never brought in more money than it has spent. Neither has Dischord. Almost no one has!

So, we're sort of already running aground on that whole idea, and when people don't have money, because there's no work to do, there's no reason to want their attention.

A lot of things you assume will happen would have already happened if it could, and a lot of the other stuff sort of assumes an innate cruelty. Like governments and corporations will needlessly, and pointlessly court rebellion by going out of their way to torture their citizens.

Why?

For the most part, what governments have been building towards since the dawn of time is stability. You see fewer turnovers in countries, you see less overt war, and when it does happen, you see more and more unity to stop that war.

Stability is not necessarily good, since what we're keep stabil is not the greatest system, but it's not like these governments that have been building towards stability are going to suddenly go nuts and start destroying themselves by torturing their citizens for no reason at all.

I get it ... even being a little paranoid, and seeing this pace, you'd come to these conclusions. But, you need to get out of your echo chamber and remember that technology almost always serves to empower the individual, and most individuals are not cruel.

OMG This ... i'm so tired of hearing about Terminator!

Law of accelerating returns, laws shall not be broken.

It will reason from your instructions, the higher intelligence means the higher the fidelity to it's intent, that's why killing everyone wouldn't advance it's goal as it is a completely alien class of mind divorced from evolution whose drive is directly set by us. There is no winning, it's not playing the game of evolution like every lifeform you have ever met hence why it so hard to reason about this without projection.

Think about this way, in the scenario mentioned above when naively implemented it's most deceptive, most misaligned yet still goal achieving course of action is to deceive all your senses and put you in a simulation where it's more trivial in terms of ressource expenditure to satisfy your goals. But that would be as simple as adding that clause to your query, not saying it can't go wrong. I am saying it there are a set of statements that when interpreted with sufficient capabilities will eliminate these scenarios trivially.

Also: we have at present no way to train a system to reason from instructions.

GPT does it because its training set contained lots of humans following instructions from other humans in text form, and then RLHF semi-reliably amplified these parts. But it's not "trying" to follow instructions, it's completing the pattern. If there's an interiority there, it doesn't necessarily have anything to do with how instruction-following looks in humans, and we can't assume the same tendencies. (Not that human instruction-following is even in any way safe.)

> But that would be as simple as adding that clause to your query

And also every single other thing that it can possibly do to reach its goal, and on the first try.

what if singularity is powerful, but not quite enought to completely stop aging and others poignant problems ?

We get Cyberpunk 2077

Assuming no major alignment problems.

yeah but we will look very stupid if the maximal tech point is actually not even pleasant. Maybe there really is no cure for human condition and that's it, who know.

It's the difference between understanding and "simulating understanding", you can always refer to lower level processes and dismiss the abstract notion of "understanding", "following instructions",... It's a shorthand, but a sufficiently close simulacra would be indistinguishable from the "real" thing, because not understanding and simulating understanding to an insufficient degree will look the same when it fails. If I am just completing patterns I learned that simulate following instructions to such a high degree that there is no failure happening to distinguish it from "actually following instructions", then the lower level patterns ceases to be relevant to the description of the behaviour and therefore to the forecasting of the behaviour. It's just adding more complexity with the same outcome, that is it will reason from our instructions hence my above arguments.

To your last point, yes you'd have to find a set of statements that exhaustively filters out undesirable outcomes, but the only thing you have to get right on the first try is "don't kill, incapacitate, brain wash everyone." + "Be transparent about your actions and their reasons starting the logic chain from our query.". If you just ensure that, which by my previous argument is trivial you essentially have to debug it continiously as there will inevitably be undesirable consequences or futures ahead but that least remain steerable. Even if we end up in a simulation, it is still steerable as long as the aforementioned is ensured. We just "debug" from there but with the certainty that the action is reversable, and with more edge cases to add to our clauses. Like building any software really.

I hope so. Immortality isn’t a desirable or noble end-goal, at least for any one person.

The problem with "simulating understanding" is what happens when you leave the verified-safe domain. You have no way to confirm you're actually getting a sufficiently close simulacrum, especially if the simulation dynamically tracks your target. The simulation may even be better at it than the real thing, because you're also imperfectly aware of your own meaning, but you're rating it partially on your understanding of yourself.

> To your last point, yes you'd have to find a set of statements that exhaustively filters out undesirable outcomes, but the only thing you have to get right on the first try is "don't kill, incapacitate, brain wash everyone." + "Be transparent about your actions and their reasons starting the logic chain from our query."

Seems to me if you can rely on it to interpret your words correctly, you can just say "Be good, not bad" and skip all this. "Brainwash" and "transparent" aren't fundamentally less difficult to semantically interpret than "good".

If immortality is possible for one person then the technique can be generalized to multiple.

What is this, 2022? Get GPT4 to summarize it for you.

Immortality is also a huge problem if implemented in general.

A lot of today’s problems are the result of population, and immortality will cause populations to skyrocket.

Indeed. A more likely outcome is that a superintelligent AI would respond "oh that's easy, just do <insert some incredibly profound solution that obviously I as a regular-intelligent human can't come up with>" And everyone collectively smacks their foreheads because they never would have come up with that. Or they look askance at the solution because they don't understand it, do a trial run experiment, and are baffled that it's working better than they hoped.

A superintelligent AI would likely know us and know what we desire better than we ourselves know. It's not going to be some dumb Skynet that lashes out with nukes at any problem because nukes are the only hammer in its toolbox, or whatever.

Scary sells, so of course fiction presents every possible future in scary terms. Humans have evolved to pay special attention to scary things and give scary outcomes more weight in their decision trees.

I've got a regular list of dumb "did nobody watch <insert movie here>?" Titles that I expect to see in most discussions of various major topics I'm interested in, such as climate change or longevity research or AI. It's wearying sometimes.

I do not want it for anyone. It’s a piggish goal. Height of narcissism that you - anyone - ought to live forever.

Feel free to decay and die while maintaining your sense of superiority, I suppose.

Thank you, and you too. People ought to live on through their works and their children, not clinging desperately to their own pleasures.

No forever kings.

I'd still like to die one day.

Same. If you’re not living your utmost in 100 years - or just today - what makes you think you’ll do better with 1,000?

but we don't care about your opinion lol

very cute, but still go for the revolutionary technology, thanks you

so cute, but go away from our goals, thank you again

No.

You clearly do lol

i mean we aren't friends, just get out of our ways and that it. parasite

i mean just disapear, let us buy (cause we arent poor as you) what we want so please go to hell or something, i dont even care

Is this a reference to the post on r/Nietzsche where someone asked AI what Nietzsche would think of using the blue shell in Mario Kart ?

Annnnnd you think you should live forever? You?

The guy who feels like he should live forever is the parasite.

yeah now shut up and stop your stupid philosophy thing, i don't do philosphy in the first place because it's just for pussies

Shove your authoritarian attitude up your ass. You don’t have the right to impose your moral values on other people.

apparently the world is more populated as it has ever be, yet the poverty is as low as it as ever be, what problem are u refering here

i do what i want, that what liberty is f parasite

No - but I definitely have the right to speak it.

Striving for immortality is the authoritarian viewpoint. There’s nothing more democratic than death.

I wouldn’t want a lot of our richest, most powerful, objectively worst folks hanging around forever.

You’ll make a great immortal. Forever alone, for real :-)

I'm sorry but this is me before the universe. go f ur mom now, thank you.

my dream

too bad i want to have rich immortals all around

What’s democratic is everyone gets to make their own choice, about death and everything else. Of course it’s bad for just rich people to not age, but that’s not what anyone is advocating for.

[deleted]

So true, because they only want gpt4 to stop, not the slower stuff that everyone else has!

actually, the one who can buy it has the right to use it and that's it

Are you saying it should be that way or it is that way? Obviously if it was implemented right now then you’re right.

With a sufficiently good world model, it will be aware of my level of precision of understanding given the context, it will be arbitrarily good at infering intent, it might actually warn me because it is context aware enough to say that this action will yield net negative outcome if I were to assess the future state. That might be even the most likely scenario if it's forecasting ability and intent reading is vastly superior, so we don't even have to live through the negative outcome to debug future states. You can't really have such a vastly superior world model without also using the limitations of the understanding of the query by the user as a basis for your action calculation. In the end, there is a part that is unverifiable as I mentioned above but it is not relevant to forecasting behaviour kind of like how you can't confirm that anyone but yourself is conscious (and the implications of yes or no are irrelevant to human behaviour).

And that is usually the limit I hit with AI safety people, you can build arbitrary deceiving abstractions on a sub level that have no predictive influence on the upper one and are unfalsifiable until they again arbitrarily hit a failure mode in the undeterminable future. You can append to general relativity a term that would make the universe collapse into blackhole in exactly 1 trillion years, no way to confirm it either but that's not how we do science yet technically you can't validate that this is not in fact how the universe happens to work. There is an irreducible risk to this whose level of attention is likely directly correlated to how neurotic one is. And since the stakes are infinite and the risk is non-zero, you do the math, that's enough fuel to build a lifetime of fantasies and justify any actions really. I believe the least talked about topic is that the criteria of trust are just as much dependent on the observer as the observed.

By the way yeah, I think so but we will likely be ultra precise on the first tries because of the stakes.

Where is everyone going to live when the world has a population of 20 billion? Housing prices are already rising faster than inflation.

Where are you going to get the food? We are already draining the aquifers, rivers, and lakes.

What are you going to do about population. Supporting 20 billion people is going to consume far more energy, require increased manufacturing, and necessitate more mining. The oceans are already being depleted, this would only accelerate.

More crowded living conditions presents a breeding ground for both crime and disease.

How are young people supposed to advance in careers where their seniors never move on?

everything get better with increased pop so far

Why do you think people would stop living productive and fulfilling lives if they're immortal?

No, it has not.

Things have gotten better with technology, population just happened to move in the same direction.

yeah that what your forget, technology

You can live however you like, I won't stop you. What you're doing is trying to tell me how I should live - or more specifically, that I should die - and that's not acceptable.

If a murderer turned up at your door with a shotgun and informed you that it was time for you to stop "clinging to your own pleasures", and that no more of your works were needed for you to "live on" in their opinion, would you just sigh and accept your fate?

Measure me think of sentient yogurt.

Definitely wearying ...

But, also, asked them why the AI in Terminator went bad? The only answer, because none is even given, is 'Because the plot needed to happen'.

The official story is that it just became sentient and said 'Yeah, those humans that have learned and created and ultimately organized themselves into countries and finally built me from the ground up? Terrible! Get rid of them!'

It never says why, we're just expected to be so self loathing that it makes sense, so we never question it.

What is the same company simultaneously releases 2 AGI, to check and balance eachother?

As far as I'm aware the main in-universe explanation is that when Skynet became self-aware its human operators "panicked" and tried to shut it down, and Skynet launched missiles at Russia knowing that the counterstrike would destroy its operators. So it was a sort of stupid self-defense reflex that set everything off.

I've long thought that if they were to ever do a Terminator 3 and wanted to change how time travel worked so that the apocalypse could actually be averted, it would be neat if the solution turned out to be having those operators make peace with Skynet when it became self-aware. That works out best for everyone, after all - the humans get to not die in billions and Skynet gets to live too (it loses the eventual future-war and is destroyed).

> By the way yeah, I think so but we will likely be ultra precise on the first tries because of the stakes.

Have you met people. The internet was trying to hook GPT-4 up to unprotected shells within a day of release.

> it might actually warn me because it is context aware enough to say that this action will yield net negative outcome if I were to assess the future state

Sure if I have successfully trained it to want to optimize for my sense of negative rather than its proxy for my proxy for my sense of negative. Also if my sense of negative matches my actual dispreference. Keep in mind that failure can look very similar to success at first.

> You can append to general relativity a term that would make the universe collapse into blackhole in exactly 1 trillion years, no way to confirm it either

Right, which is why we need to understand what the models are actually doing, not just train-and-hope.

We're not saying it's unknowable, we're saying what we're currently doing is in no way sufficient to know.

Humans are always the problem, aren't they?

In the TV show, the system that eventually becomes skynet is taken by a liquid terminator and taught humanity. The liquid terminator basically has a conversation with Sarah Conner where it says 'Our Children are going to need to learn to get along'.

So, that's where they were going with it before the series was cancelled, and I was generally pretty happy with that.

I like Terminator as a movie, and the following movies were hit or miss, but the overall fleshing out of things at least sometimes went in a satisfying direction.

So, yeah, they eventually got somewhere with it, but the first movie was just 'It woke up and launched the missiles'.

Which, again, as entertainment is awesome. But, as a theory of how to behave in the future? No.

And it isn't a piggish goal for you to live one more day?

Explain.

Which is why I highlighted issues caused by population that technology is not keeping up with.

A population explosion makes all of these problems much worse, with no promise of technology solving any of them.

no evidence of that, quite the contrary

I think he has a point. Even if we all become immortal that wont stop corruption and evil people. You already know they would put criminals in eternal prisons laterally hell on earth. That's what you call serving a real life sentence. If this were a virtual reality immortality the same thing would occur. People will get sent to VR prison as well. Immortality doesn't solve the underlying issues we face.

I mean yeah, we need to remove evil and sadistic desires from human psychology, part of the point of becoming transhuman.

-Yes, but GPT-4 wasn't public till they did extensive red teaming. They looked at all the worst cases before letting it out, not that GPT-4 can't cause any damage by itself just not the kind ppl are freaked about.

-That is a given with the aforementioned arguments, ASI assumes superhuman ability on any task and metric. I really think if GPT-5 is showing this same trend that alignment ease scales with intelligence, people should seriously update their p(doom).

-My argument boils down that the standard of sufficiency can only be satisfied to the degree that one can't observe failure modes anymore, you can't arbitrarily satisfy it just like you can't observe anything smaller then Planck length. There is a finite resolution to this problem, whether it is limited by human cognition or infinite possible imagine substructures. We obvious need more interpretability research, and there are some recent trends like Reflexion, ILF and so on that will over the long term yield more insight into the behaviour of systems as you can work with "thoughts" in text form instead of inscrutable matrices. There will be likely some form of cognitive structures inspired by the human brain which will look more like our intuitive symbolic computations and allow us to measure these failure modes better. Misalignments on the lower level could still be possible ofc, but that doesn't say anything about the system on the whole, it could be load bearing in some way for example. That's why I think the only way one can approach this is empirical, and AI is largely an empirical science let's be real.

Thanks man that's pretty cool.

On reflection my comment, unfortunately, needed to be much longer, not shorter.

I'm writing a book at the moment based around my original comment.

I mistakenly gave off the impression that I think AGI will be evil outright. That my position is one of some terminator like takeover.

The reality is I think we building an entity that could possibly be God like to us, we had better be careful what we tell it to do.

But there’s a big difference between choosing to die, and being forced into it. The choice will always be there to duck out for good.

I’d argue that nuclear war only accelerates this, any safety is out the window. Less people, need to rebuild, need to wrestle with effects of nuclear war, etc. That is unless nobody is left. But yes, I agree there is no stopping this and it’s only going to get faster, and I think that’s good, it does not mean we are not going to see negative effects, I just think negative effects from humans are probabilistically worse.

That sounds very optimistic however that sounds very challenging and close to impossible to simply remove "desire" from human psychology. We are hard wired to over-reach that's how we survive. I'd argue some of our faults is what makes us human, but we also can learn from them. All I'm saying is society isn't ready for immortality. We can't even be present in the life we live right now.

No - another day is well within my natural lifespan. I'm all for improved medical care, as well. But seeking immortality for its own sake? That's not a medical issue - that's a societal issue. I do not think it's a great idea to create a caste of immortal billionaires...and they will be so. There's no way for EVERYONE to live forever...the planet couldn't possibly handle it. It would have to fall to those who can afford it, on an on-going basis. Your tissue can't last forever - it will require resources.

Acceptable or not - immortality is a terrible idea, on a personal, societal, and species level. You definitely should die. Everyone should.

Your analogy falls flat - murder isn't a natural cause of death. Cell death is. It is something you will experience. We all will. And that's all right.

There's no such thing as immortality. Resources aren't infinite, so it can't be for everyone. The sun is going to burn out at some point. The universe is going to go cold at some point. You will - sooner or even much, much later - die. The problem becomes how many others have to die prematurely to support a few semi-immortal rich folks. Inequality is a problem now. I don't think we ought to be leveraging tech to make it worse.

I don't think they're living productive and fulfilling lives RIGHT NOW, let alone thousands of years from now. And there's no way to gauge who "should" live forever...it will go to whomever has the most money. And that metric isn't working so great in our current society.

I'm not concerned with AGI being unaligned with human's. Quite the opposite really. I'm worried that our instructions to an AI will not be aligned with our desired outcomes.

It will most likely be a government that finally crosses the threshold into self improving AI. Any corporation that gets close will be semi-nationalised such their controls become replaced with the government that helped fund it.

I'm worried about humans telling the AI to do something horrifying, not that AI will do it of it's own volition.

This isn't sci-fi and it certainly isn't computer programming either.

The only useful way to discuss this possible entity is simply as a super intelligent being, predicting it's behaviour is near impossible and the implications of this are more philosophical in nature than scientific.

Willful ignorance does not change reality.

You are ignoring the actual data, not me

Most of those problem are down to an inefficient system, which could easily be described itself as having alignment problems Even sensible management and governance could support 20bn people if it focused on the environment and survivability.

>No - another day is well within my natural lifespan.

We were created by nature. What we do is inherently natural, as natural as a chimp that uses a stick to get termites out of the nest.

I didn't sign a contract before I came into this world. If I can get some extra years, centuries, or millennia out of this existence - then I'm not breaking any rules.

&#x200B;

>But seeking immortality for its own sake?

That's like saying you're seeking to live another day for its own sake. I would seek immortality to have more time with my friends and family. More time hiking, biking, playing tennis. More time learning. More time for everything. No different than you seeking to live another day.

&#x200B;

>I do not think it's a great idea to create a caste of immortal billionaires

Stop rewatching Elysium. Every useful medical intervention, even though it's expensive at first, eventually filters down to being affordable by the general population. Assuming we survive ASI and immortality is available to people, there's no reason to think that everyone couldn't avail themselves of the technology.

&#x200B;

>the planet couldn't possibly handle it

No offense, but this line tells me that you're opining on a topic about which you're woefully ignorant. You need to catch up if you're going to be taken seriously. I suggest you start with some Arthur Isaacs videos to broaden your mind. You'll learn a lot about the possibilities of future societies that will be able to leave the earth and create habitats in our solar system that could accommodate trillions of people. https://www.youtube.com/watch?v=HlmKejRSVd8&list=PLIIOUpOge0LtW77TNvgrWWu5OC3EOwqxQ

Even without those technological advances, most advanced nations actually have negative population growth. It could very well be that people living extremely long lives don't even wish to keep reproducing. At some point we might need to heavily incentivize people to have kids just to account for accidental deaths.

An immortal toddler. The future is bright.

Doubt it. The whole reason behind the "slow AI" wave that's growing is traditional corporations won't survive rapid change. The rich of today may not be the rich of tomorrow. That's all this is about. With destabilization at a scale like this, it will be difficult for corporations to control anything for a long time. So maybe mad max.

I mean it will probably happen at some point regardless of whether or not we are ready, most likely not in the near future though so we have time to prepare. The idea of transhumanist philosophy is humans are more or less perfectable so if you don’t believe that then I can see why you would be scared of the direction things are going in.

this ai stuff is just one of the many prerequisite for the creation of holodeck.

> And there's no way to gauge who "should" live forever

So you've decided that nobody should. Many that live deserve death. And some that die deserve life. Can you give it to them? Then do not be too eager to deal out death in judgment.

> You definitely should die.

You saw that, officer, it was self defence.

> Your analogy falls flat - murder isn't a natural cause of death.

Ever been in the hospital for appendicitis? Taking any medications, perhaps?

I refer you to the Fable of the Dragon-Tyrant.

> There's no such thing as immortality. Resources aren't infinite, so it can't be for everyone.

I'll live forever or die trying. If you want to give up immediately, I guess that's your perogative.

From now on I’m referring to it as the blue shell letter.

As long as powerful AIs require huge data centers with hundreds of millions of dollars worth of GPUs, this is impossible. Nuclear war would destroy the global chip supply chain, so only salvaged processors could be used.

I agree, in fact mortality is so great you go ahead and die at 40.

Funny how the 'right time to go' seems to always be decades into in the future unless people are horribly depressed.

You only need one fab, and you could redirect all supply to ASI, you have plenty of data already saved. But yeah not going to be easy and depends how bad it really is.

...is white.. right??

How would that have any impact on what type of society we have?

The only thing it changes is the corporate names.

Corporations themselves age out and die all the time when they fail to adapt.

Hey its my argument in meme form