The real issue is that we live in an economic system where people are exploited for labor, and in turn they buy products and services made with their own labor (and another class get to profit from it).
If we introduce AGI but keep the system, people will be unemployed. If people aren't employed (and instead machines do their jobs), then they can't buy stuff. The whole system crumbles.
But it's possible that AGI will be disruptive enough to completely change the system. Let's hope it's a change for the better.
I see an impending intersection of three phenomena, with potentially disastrous results for society:
* Social media is decreasing the average attention span. TikTok is accelerating the trend of people not having time to look past a soundbite or headline in an endless scrolling feed. Intellectual depth and critical thinking vanishes.
* AI deep fakes make truth unknowable. Given the above, the majority of people will take these at face value, or they will give up, because "who can even know what's true anymore?"
* UBI (required because of the coming labor automation revolution) will keep everyone complacent. I'm happy, why would I care who gets elected, or what the government does, as long as I can still buy stuff and eat well?
The logical conclusion is that we fully transition from citizens into a herd of consumers with goldfish attention spans. Voter participation rates plummet. The populace is no longer able to hold government accountable.
Assume the existence of a large scale Star Trek Replicator that can almost instantly create anything.
There are only two possibilities that result:
1) We now live in a post-scarcity society where everyone self-actualizes and no one wants for anything.
2) We now live in a society where the small % of the population that owns the Replicators self-actualizes and wants for nothing while the remaining 99.9% of the population can f** off and die.
While we can get to a post scarcity society where people can live for free without a job, there are still going to be economies around "liberal arts". You cant realistically say "hey replicator, give me a usb filled with music that I like". You would have to find out which music you like, and random search on this is not really enjoyable, which would then means that there is economic opportunity for discovery, e.t.c.
Sounds like we arrive at point #1 in any case, there's just a question if a mass genocide happens in between. Probably depends on how gradual the transition ends up being.
If we introduce AGI but keep the system, people will be unemployed. If people aren't employed (and instead machines do their jobs), then they can't buy stuff. The whole system crumbles.
But it's possible that AGI will be disruptive enough to completely change the system. Let's hope it's a change for the better.