The sperm cells themselves are very short-lived. However, they do originate from spermatogonial stem cells that keep on replicating theough all your life.

In a cartoon, these stem cells divide into two cells. One of these daughter cells (cell A) keeps being a stem cell and the other not (cell B), begining a chain of divisions into further daughter cells that end up being a bunch of sperm.

New mutations appearing on cell B will only be found in one "round" of sperm. Mutations appearing in cell A are there to stay in all future rounds.

Not really.

The frequent replacement of sperms cells is “helpful” in a sense to reduce the incidence of chromosome abnormalities like Down syndrome. For women, since they are born with all the eggs in their body, those eggs age with them and with that age it is thought that the chromosomes get kind of “sticky” and can have trouble dividing easily. This is why Down syndrome can be seen more frequently with older moms.

The same is not true to sperm. However, there are risks associated with advanced paternal age and those risks are associate with mutations. For example order dads have a high chance of having a kid with achondroplasia. This is thought to be related to the fact that the sperm of an older man has undergone more cell division leading to more chance of these small mutations.