They usually died of malnourishment as children. But there's plenty of cultures who have rice or corn based diets.

Also, gluten in everything as a cheap filler/stabilizer is a fairly recent development.