Weathering definitely happens at different speeds, depending on the type and intensity of weathering. There mainly are 3 types:

• physical (water and wind move sand and small rocks around, they become smaller through friction; material reacts to temperature differences by extending/shrinking, making way for water to intrude through cracks; also water then potentially freezing, extending and breaking rocks)

• chemical (water reacts with minerals, resulting in the emergence of more or less potent acids)

• biochemical (mainly driven by organic acids, created by decaying organisms, root secretions, huminic acids)

These three interact with each other. Weathering intensity is highest near the equator, because of basically all the life that's happening there. Great biodiversity + heavy precipitation year round means lots of fuel for weathering.

Interestingly, there's also the concept of relief energy - weathering in great heights happens quicker simply because of the greater potential/gravitational energy.