We live in a racist and classist society and we're all marinating in it 24/7. You aren't exempt; everyone has absorbed racist and classist beliefs just by existing.

Until that changes, it's what we do when we learn about the racism and classism inherent in an idea or a phrase that defines us. Now that OP is aware that it's racist and classist, they can choose not to use it anymore and can "spread the knowledge" to others.