What heritability actually shows is percent of variance is accounted for by genetics/inborn factors. How much variance there is can be environmental - that is, environment that is shared among all individuals in the study population, not just family environment. E.g. if weight was 100% heritable then genetic profile A would be 20th percentile and genetic profile B would be 90th percentile no matter what, but in 1920 that might have meant 110 pounds and 200 pounds, whereas today it might mean 130 pounds and 500 pounds. The actual weight of an individual still cannot be determined only by genetics.

Height is a good example to understand the counterintuitive math of heritability: it has become much more heritable over time because it's much more rare for people to experience malnutrition that impedes them from reaching their maximum genetic potential. The genetic pool hasn't changed significantly, and it has become more "genetically determined," but the increase in average height or the increase of height in subsequent generations of a family is very much due to environment.