Every society, certainly every powerful society, seems to be confident in its moral superiority. Military conquest, enslavement, and mass murder were apparently considered normal actions for most of history, so the real question is, why did ideological opposition to colonialism develop? It probably partly comes out of political opposition due to the unequal distribution of costs and benefits in the colonizing society, but I would guess it’s mostly from people applying the ethics they’ve learned in highly pacified cores to the world in general.