Best GPUs for pretraining roBERTa-size LLMs with a $50K budget, 4x RTX A6000 v.s. 4x A6000 ADA v.s. 2x A100 80GB Submitted by AngrEvv t3_11vb220 on March 19, 2023 at 4:17 AM in deeplearning 7 comments 18