Regarding ChatGPT, I believe OP is frustrated not by the Transformer architecture but by the improvements made in the inference functionality.

That’s the real “black box” of GPT style LLMs and the least open