In addition to what's been mentioned as far as political power, it's also worth noting the role of capitalism and industrialization. As capitalism expanded into a global market and technology made it possible to process and produce things like textile products more quickly, you see the plantation system develop into a labor system, and white plantation owners relied on unfree labor to continue growing that. It's worth noting that even a lot of anti-slavery white Americans pre-emancipation are against enslaved labor not because they necessarily care about enslaved African Americans, but because they feel that it's a threat to free white labor.

Some sources to check out if you're interested in exploring the connection between capitalism and slavery:

Slave Country

Slavery's Capitalism

The Half Has Never Been Told