For the Colosseum (Flavian Amphitheater) and many other structures it was because the buildings were no longer necessary, and the materials used in their construction could be better used elsewhere. Many cathedrals and other public works were created using the stones of ancient buildings. If you look closely at the Colosseum, you'll see thousands of little holes. These are from people who harvested the iron clamps the Romans used to hold bricks together.