Viewing a single comment thread. View all comments

maskedpaki t1_j8by3r7 wrote

This is like 2 weeks old. If it really does surpass gpt3 with under a billion parameters then why isn't this on headlines.

11

d00m_sayer t1_j8bzbnm wrote

Because people cannot use it yet 🙄

26

maskedpaki t1_j8c0eso wrote

I've seen so many things like this that actually end up surpassing gpt3 on some narrow benchmark with more optimised prompting rather than just being a better model overall

I hope I'm wrong this time

17

94746382926 t1_j8c1t4r wrote

Yeah we need more benchmarks.

5

beezlebub33 t1_j8d62pw wrote

Benchmarks are really hard and expensive. And they are not fun or exciting for the people involved; the groups that make them really deserve more credit.

1

gay_manta_ray t1_j8c1uv8 wrote

this benchmarks seems pretty comprehensive

3

maskedpaki t1_j8g2v6v wrote

no actually seems like a pretty narrow science benchmark

​

if you told me the MMLU 0 shot was higher than 175 billion gpt 3.5 with under a billion parameters then id be absolutely shocked

1

Ok_Criticism_1414 OP t1_j8bywbq wrote

because of ChatGPT hype ? Who knows. I think open AI already did it, the just dont showing to the puclic. Main thing i guess that Amazon made a different aproach integrating two modalities by prefintuning to be multimodal. You can read in the paper. + Looks like language + visual context gives a huge boost. But it already being done by Flamingo model so i gues the first is crucial.

2