BullockHouse t1_jdidje0 wrote on March 24, 2023 at 4:37 PM

I'm curious if it can be instructed to play minecraft in a keyboard only mode simply by connecting a sequence of images to key stroke outputs.

wyrdwulf t1_jdikhuo wrote on March 24, 2023 at 5:21 PM

They had another model do that already.

OpenAI: We trained a neural network to play Minecraft by Video PreTraining (VPT) on a massive unlabeled video dataset of human Minecraft play

BullockHouse t1_jdil2ok wrote on March 24, 2023 at 5:25 PM

I'm familiar! I'm curious though if it can generalize well enough to play semi-competently without specialized training. Has implications for multi-modal models and robotics.

Art10001 t1_jdihrod wrote on March 24, 2023 at 5:04 PM

Probably. And if not, certainly someday.