I am doing the same thing as you. I am currently playing with gpt2 since it’s extremely small. Then when I am comfortable I plan to play with gptj or other ~7b models. Then finally I kinda want to try something with a 20b model as a final big project maybe since I saw you can fine tune it on 4090.

I vote for 2030-2040 if you put an AI into my body nobody who know me would notice any difference. I.e. he can fake me perfectly. But 2030 I am sure many people will start to at least believe there is already agi.

That's not how capitalism works. To produce chatgpt they need a lot of money for a huge GPU farm, which needs to be invested by people who expect profits from it. If we want everything to be open sourced then chatgpt as it is now probably wouldn't be possible at all. But anyway I think basic theoretical breakthroughs like a new architecture for AI will still be shared among academia since those aren't directly related to money. Hopefully, it would just be the detailed implementation of actual products that aren't open source.

There is a reason why the advanced countries in the world are capitalism countries. Money is mechanism to control long term advancement of pretty much anything in our society. I don’t think anybody should be surprised AI development can’t be forever fully open sourced. Maybe some day it will, but certainly not before some major (global) social reformation.