Viewing a single comment thread. View all comments

Even_Tangerine_800 t1_is9zy95 wrote

Reply to comment by Co0k1eGal3xy in [R] Mind's Eye: Grounded Language Model Reasoning through Simulation - Google Research 2022 by Singularian2501

This is what I got: GPT-3 Answer.

Apparently, the model arrives at the wrong answer without mentioning the air resistance. I have tried many times the results are consistent.

Considering the free fall rules should be encoded in some text books (which should have been included in the pre-training datasets), these results are even more striking to me.

7

Co0k1eGal3xy t1_isa0dvo wrote

The heavier baseball falling to the ground faster is the correct answer. Maybe you misread my post?

It is a shame none of them mention air resistance.

3

Even_Tangerine_800 t1_isa18wf wrote

Are the questions as simple as a = F/m = mg / m = g?

Anyways. If humans put effort into optimizing a tool for accurate simulation, we can treat it more like an alignment problem rather than pure scientific judgment.

You can update the knowledge in the physics engine if you want.

3

Co0k1eGal3xy t1_isa1gbn wrote

Oh I agree 100%. This paper is fantastic! (and it's an easy fix)

I definitely want to see further research in this, but the comparison they show here is probably not the comparison they wanted to show haha.

1

Lajamerr_Mittesdine t1_isa34ib wrote

All the answers are incomplete because they don't provide the assumptions necessary to arrive at a complete solution.

A more complete answer would look like this.

>Assuming just gravitational forces both the lighter and heavier baseballs both would fall at the same rate and then reach the surface at approximately the same time. This can be impacted however by additional forces that may be present such as an atmosphere providing additional resistances based on the surface area, density, and total mass of each object.

Though even that is an incomplete answer.

1

Co0k1eGal3xy t1_isa44ws wrote

>because they don't provide the assumptions necessary to arrive at a complete solution.

I agree, but when atmosphere it not mentioned, the default should be updated to STP (0°C temperature and 101.325 kPa pressure) in future.

2