FirstOrderCat t1_itaeypy wrote on October 22, 2022 at 3:03 AM

Reply to comment by AsthmaBeyondBorders in U-PaLM 540B by xutw21

this race maybe over.

On the graph guy is proud of getting 2 points in some synthetic benchmark, while spending 4 millions TPUv4 hours = $12M.

At the same time we hear that Google cuts expenses and considering layoffs, and LLM part of Google Research will be the first in the line, because they don't provide much value in Ads/Search business.

AsthmaBeyondBorders t1_itagbfz wrote on October 22, 2022 at 3:15 AM

This model had up to 21% gains in some benchmarks, as you can see there are many benchmarks. You may notice this model is still 540B just like the older one, so this isn't about scale it is about a different model which can be as good and better than the previous ones while cheaper to train.

You seem to know a lot about Google's internal decisions and strategies as of today, good for you, I can't discuss stuff I have absolutely no idea about and clearly you have insider information about where google is going and what they are doing, that's real nice.

FirstOrderCat t1_itahazn wrote on October 22, 2022 at 3:25 AM

>This model had up to 21% gains in some benchmarks, as you can see there are many benchmarks

Meaning they received less than 2 points in many others..

> it is about a different model which can be as good and better than the previous ones while cheaper to train.

Model is the same, they changed training procedure.

> You seem to know a lot about Google's internal decisions and strategies as of today

This is public information.

justowen4 t1_itau79p wrote on October 22, 2022 at 5:45 AM

Epic commenting you two. The winner is….. AsthmaBeyondBorders !