thesupernoodle t1_jcsll6u wrote on March 19, 2023 at 6:20 AM

Reply to comment by FirstOrderCat in Best GPUs for pretraining roBERTa-size LLMs with a $50K budget, 4x RTX A6000 v.s. 4x A6000 ADA v.s. 2x A100 80GB by AngrEvv

Sure; but the broader point is they can optimize their need with some cheap testing - is the model big enough such that is wants the extra ram of an 80Gig A100?

thesupernoodle t1_jcsj2iw wrote on March 19, 2023 at 5:47 AM

Reply to Best GPUs for pretraining roBERTa-size LLMs with a $50K budget, 4x RTX A6000 v.s. 4x A6000 ADA v.s. 2x A100 80GB by AngrEvv

For maybe a few hundred bucks, you can test out the exact configurations you want to buy:

https://lambdalabs.com/service/gpu-cloud

You may even decide that you’d rather just cloud compute, as opposed to spending all that money upfront. It would only cost you about 19 K to run 2xA100 in the cloud for 24/365 for a solid year. And that also includes electricity costs.