Horace He made a nice thread on this when GPT-4 first came out. Realistically this is expected - within the short time span, there isn't much else you can do to improve the model performance other than increasing size of training data, which resulted in data contamination.

I expect the next "big thing" to be some of self-correcting output, or better chain-of-thoughts reasoning.