It would also depend on how the colonization was made, the scale of atrocities and who the victims were. And obviously who the aggressor was and their current reputation.

As an example, Leopold's "Kongo Free State" was condemned internationally in the 1890s when conditions of forced labor, torture and executions become known in Europe in open letters such as this:

George Washington Williams, an African-American journalist, pastor, historian, lawyer, and Civil War veteran, after visiting the Congo in the spring of 1890. Hoping to witness firsthand Leopold’s alleged philanthropic works, Williams instead left Africa outraged and disillusioned. He wrote Leopold shortly after, “in plain and respectful language,” protesting how Congolese were swindled of their lands and brutalized by agents of the Congo Free State, including Henry Morton Stanley. He lashed out at Leopold for allowing kidnappings, coerced labour, torture, and wanton murder.

More well known critics came in the form of Joseph Conrad, writer of The Heart of Darkness, and later Mark Twain. Both these authors contributed in telling the (white) world how bad things really were in the Congo. The Casement report later verified the atrocities and were taken quite seriously, resulting in the dissolving Leopold's property and creating the Belgian Colony.

Note that many of the detractors and condemners did not disagree with colonialism, but reacted because Leopold's reign went too far and caused too much suffering that even your staunchest "white man's burden" racist imperialist could stomach it.

Still, even after this debacle most Europeans believed in Imperialism as a way to spread culture, decency, trade, and prosperity to regions and peoples they believed to not have it.