jason_bman t1_je9unzr wrote on March 30, 2023 at 1:50 PM

Reply to comment by MassiveWasabi in Microsoft research on what the future of language models that can be connected to millions of apis/tools/plugins could look like. by TFenrir

Do you know if the example Figures are hand-typed by the researchers? For example, there is a prompt in Figure 9:

Human: I hope to eat an apple and drink a cup of milk.
Can you please pick them up from the fridge and put
them on the kitchen table?

TaskMatrix.AI: Sure, I can help you with that.
robot_go_to("fridge")
robot_pick_up("egg")
robot_go_to("kitchen table")
robot_put_down()
robot_go_to("fridge")
robot_pick_up("milk")
robot_go_to("kitchen table")
robot_put_down()

Wondering if "egg" is just a typo from the research team. Seems like an error that a large LLM would not make.

MassiveWasabi t1_jeb67h2 wrote on March 30, 2023 at 7:07 PM

I looked into that just now and my conclusion is that there may be some translation issue between the researchers and the AI. The researchers are all Chinese and I can see some other simple English mistakes, so I'm not sure if they were using something for translation or if they were just typing in English. Maybe they did all of the research in Chinese and then translated for us to read the paper. I don't really know, though.

jason_bman t1_jecvrql wrote on March 31, 2023 at 2:14 AM

I actually had the same thought given the grammar mistakes. Still an awesome paper!