>1) How long will it matter that the source of our content is human? At some point I think there will be a gesture towards considering AI a comparable mind and asking them for their experiences and sharing in the joint qualms of existence. I do think that there will be a rubber band snap back of sorts whenever we realize they're smarter than the collective whole of humanity and then our ability to relate will be minimal. Also again, the trust factor of being exploited goes up dramatically.. before I get that out of your mouth

I think that one key issue here is that a true general intelligence (the comparable mind you speak of) is probably a very long way off. ChatGPT and related AIs can simulate talking about qualms of existence, but its true existence is as something which generates plausible responses to text prompts based on its training data. If there's anyone in there with a subjective experience (most likely and hopefully not), it is vastly different from ours.

And if/when we do create an artificial general intelligence there are far more serious things to worry about than whether some person/corporation is exploiting you. Hopefully whoever does it has somehow also managed to figure out all of the AI safety problems to avoid a guaranteed apocalypse. It's a surprisingly difficult problem, and is also very entertaining to think about.

We definitely don't want to find out the hard way what happens when a superintelligent stamp collector AI figures out that it can get the most stamps by overthrowing human governments and diverting all production to stamp printing, or when it finds that humans are a good source of raw materials...