That's not a whole lot of data and doesn't compare to the gargantuan amount of data already on the decade generated over decades.

The current transformer model scaling will hit a wall soon due to lack of training data.