Yeah, the real danger would be hardware. The software can be stolen or whatever, but if a strong AI requires a building full of computers to run (and at first it likely will), then it will be limited to the control of the few.

But, stories like Altered carbon really don't make sense once you have strong AI.

Why do practically any of the people have jobs? If we had strong AI, surely we could build a dumb robot that can do pretty much whatever a dumb human can do, but without any need to pay them? I'm not saying fully AI autonomous robots, but it wouldn't take much from where we are to have robots that can do all the world's jobs.

So, what do we do with the people then? In a world where technology can free them from having to do any work, you have to imagine some kind of inexplicably cruel person at the top, not only hoarding wealth, but purposely doing things to hurt those beneath them.

In short, AI could easily solve the issues of wealth inequality and create a world where no one suffers. We already produce more food than the world needs, yet people starve? It's not a matter of work, or resources. It's a matter of organization.

So, with an AI organizing the world's resources, you'd have to make the conscious choice to force others to starve. The AI would say 'So, if we just start this way of distributing food, we can end starvation', and someone would have to say 'No, don't do that. I want to see them starve, even though the other option means just throwing that food away.'

Of course some people, maybe .001% of the population, are exactly like that, and those people might even rise to the top. But, it only threatens them to make needless enemies of the rest of the population, instead of letting the AI solve their problems and be hailed as a god.

I just don't really see a way that the current system, based on scarcity, continues in a world where there's no need for scarcity. Once a machine can build machines that can do our work for us, and organize our resources, there's no reason for anyone to work or starve, and anyone that tries to create a reason for that is just making needless trouble for himself.

Cyberpunk stories, like Altered Carbon, or Elysium all fall prey to the same issue. They imagine a single technology advancing, and the ramifications of that, while they forget what the ramifications of all the other technologies advancing at the same time would be.

In both, you have this absurd world where the 'rich' are rich just because they happened to be born in the right circumstances, and 'rich' really only means the ones who hold down everyone else, because you can manufacture bodies to do the work, and heal human bodies infinitely, and mine asteroids if you want to for resources.

There's no scarcity, and the 'rich' are really just bullies, there to torture the people by denying them access to what costs them nothing.

It doesn't make any sense that a world like that would exist for any amount of time before everyone in the bully's organization turns on them.