It's not just the cost of training. It's the availability of quality data. Meta/Google/Facebook/Microsoft are all on the forefront of this due to the access to data.

I see no problem with this. What could possibly go wrong? lol

Yes. I don't believe AI exhibits this either.

Google invented Transformers. It was released in 2015 I think. It was the first model focusing on massive attention mechanisms.

So compounding exponential is just exponential I think. But I think Jack Clark may have imagined something like E^(E^x))

We have to pay homage to algorithmic improvement too. Neural Network Models like Transformers, Pretrained Transformers, Generative Adversarial Networks were all introduced in 2010-2020 decade and without those models, current changes would not be possible. So data, yes, processing power, yes, but models too.