Viewing a single comment thread. View all comments

mishap1 t1_j2cd6gg wrote

Reply to comment by DaffyDogModa in There's now an open source alternative to ChatGPT, but good luck running it by ravik_reddit_007

$6M in A100 GPUs plus all the hardware necessary to run them. Seems totally manageable.

61

Admirable_Royal_5119 t1_j2cj5co wrote

80k per year if you use aws

20

shogditontoast t1_j2cps4g wrote

Wow I’m surprised it’s so cheap. Now I regret working to reduce our AWS bill as that 80k would’ve previously gone unnoticed spread over a year.

24

username4kd t1_j2coq39 wrote

How would a Cerebras CS2 do?

3

SoylentRox t1_j2f9pw8 wrote

I think the issue is the cerebras has only 40 gigabytes of SRAM.

Palm is 540 billion parameters - that's 2.160 terabytes in just weights.

To train it you need more memory than that, think I read it's a factor of 3*. So you need 6 terabytes of memory.

This would be either ~75 A100 80 GB GPUs, or I dunno how you do it with a cerebras. Presumably you need 150 of them.

Sure it might train the whole model in hours though, cerebras has the advantage of being much faster.

Speed matters, once AI wars get really serious this might be worth every penny.

5

nickmaran t1_j2e09em wrote

Let me get some pocket change from my Swiss account

2