I'm not sure what you mean about the city being less diverse, but there were gangs that beat up on minorities back then, notably the Dot Busters and the Bones, who were violent and despicable. Most people can't jump up and move when these things happen, so I don't think there was a mass exodus of any type. Aside from that, there was also run of the mill robberies, car thefts, etc.