it's an open question and lots of interesting work is happening at a frenetic pace here

• Language Models Can (kind of) Reason: A Systematic Formal Analysis of Chain-of-Thought https://openreview.net/forum?id=qFVVBzXxR2V
• Emergent Abilities of Large Language Models https://arxiv.org/abs/2206.07682

A favourite discussed recently:

• Theory of Mind May Have Spontaneously Emerged in Large Language Models https://arxiv.org/abs/2302.02083