There's a few factors at play.

One is, like you mention, how long people have lived there. So for a thing like a temple, you usually get it to last a bit longer as most people will venerate it and typically will want to make sure it lasts, especially those who worship or recognise that deity (though people like Herostratus have existed). But as time passes, and that religion is replaced, or new people move in that don't respect or recognise that god they might not care for the temple at all, or might deliberately destroy it, or simply start to use the stones and other such things to build their own things. Or they might decide it's a great place to store gunpowder and the Venetians might decide to fire upon your gunpowder supplies to keep you from firing back at them, you know, hypothetically.

But also you need to look at the geography and geology of Greece. There's a lot of hills and mountains, this has a lot of benefits for living there, but it also means there's a lot of stuff constantly going down from higher points of elevation to the lower parts where you might find more settlements. This can be just basic runoff, to landslides, to complete shifts in the land itself from earthquakes and just general plate tectonics. Look for example at how much the area around Thermopylae has changed https://upload.wikimedia.org/wikipedia/commons/c/cf/Thermopylae_shoreline_changes_map.png

You also get even more violent events like volcanoes. Which is why places like Pompeii got covered completely. Of course the people living there knew it was there, and their descendants would hear about it and learn it was there, but overtime that direct knowledge was lost and while we had texts they weren't maps, and still only a small population would have been reading and familiar with these stories. The volcano still would keep up its attack and burry it further and further and further down overtime, as well as just that general passage of time.

And that comes back to the first point, how many people have continued living there. New homes, new buildings, new structures, etc. are built over places eventually too. And while there certainly have been times where it's been decided "well, we need to tear down your home and excavate this area" it's generally not a method that's utilised a lot.

And of course this isn't even describing every way things get lost over time: there's a lot more to it, but hopefully it gives you a sense of it all. And this means that there's a lot of cool stuff out there to be discovered. Some most of it probably never will be found. We still have mentions of entire towns and cities that we just don't know where they are. It could be a simple thing as "well, this source calls this town that we already know about by this weird name that was rarely used, it's not lost" some others are "yeah this place is just missing entirely." and it could be under a bunch of houses right now.