Yep. The unspoken knowledge. When you are raised around high status people, you benefit from their connections, from learning upper class sociolect, mannerisms, style of dress, hobbies, even gait.

Me and my friend noticed years ago that poor men, unlike all other people in society, tend to swagger instead of walk. If you gave a poor man a good haircut and nice clothing that fit, you would still be able to tell he's poor because of his gait.