Best GPU for deep learning

Submitted by somebodyenjoy t3_z6kr2n in deeplearning

I currently have a 1080ti GPU. I took slightly more than a year off of deep learning and boom, the market has changed so much. I'm looking for advice on if it'll be better to buy 2 3090 GPUs or 1 4090 GPU. I run into memory limitation issues at times when training big CNN architectures but have always used a lower batch size to compensate for it. I will also be using hyperparameter tuning, so, speed is a factor. How much lower will training speeds on smaller models be if I use 2 3090s? If I can use lower batch sizes, will I be better off going with a 4090? Are there any better options that I missed?

I haven't yet done multi-GPU training, so will I be able to use my current PC in case I do need a bigger memory? This way I'll be able to buy a 4090, and just use this guy for when I need a bigger memory

​

This is my current PC build, I'm guessing I'll have to upgrade almost everything

​

​

4

Comments

You must log in or register to comment.

--dany-- t1_iy1zq6n wrote

3090 has NVLink bridge to connect two cards to pool memories together. Theoretically you’ll have 2x computing power and 48GB VRAM to do the job. If VRAM size is important for your big model and you have a beefy PSU then this is the way to go. Otherwise just go with a 4090.

If you don’t need to train a model frequently, colab or some paid gpu rental services might be easier for your wallet and power bill. For example it’s only about $2 per hour to rent 4x RTX A6000 from some rentals.

5

CKtalon t1_iy2n56h wrote

NVLink doesn’t pool VRAM no matter what Nvidia’s marketing says. I have NVLink. It just doesn’t.

8

somebodyenjoy OP t1_iy20svg wrote

Hi, thanks for your reply. So 2 3090s will be faster than one 4090, correct?

1

--dany-- t1_iy2149l wrote

Not too much by some benchmarks. So speed is not your point here. Your main concern is if the model and training data can fit your VRAM.

6

somebodyenjoy OP t1_iy23k2m wrote

I do hyperparameter tuning too, so the same model will have to train multiple times. More times the better, as I can try more architectures. So speed is important. But you’re saying that 4090 is not much better than 3090 in terms of speed huh

1

--dany-- t1_iy29wqs wrote

I’m saying 2x 3090s are not much better than a 4090. According to lambda labs benchmarks a 4090 is about 1.3 to 1.9 times faster than a 3090. If you’re after speed then a 4090 definitely makes more sense as it’s only slightly slower but is much more power efficient and cheaper than 2x 3090s.

4

VinnyVeritas t1_iy2nq1w wrote

Just get two 4090 and power limit them to 350 watts using "nvidia-smi", there's almost no performance loss and you don't need NVLink anyway to do multi-gpu training.

3

CKtalon t1_iy2n7t0 wrote

Get the 4090. Besides you only have 32GB ram. Feeding 2 GPUs with data can be a bottleneck.

2

thefizzlee t1_iy38vtp wrote

I'm gonna assume the Nvidia A100 80gb edition is out of your budget but that is the gold standard for machine learning, they're usually deployed in clusters of 8 together but one is already better than 2 3090s for deeplearning.

If however you want to choose between 2 3090s or a 4090 and you're running into vram issues I'd go for the dual 3090, clustering gpus is very well supported in machine learning so you'll essentially getting double the performance and from what I know 1 4090 isn't faster than 2 3090s plus you'll double your vram

Edit: if you want to safe a buck and you're software supports it you could also look into the new Radeon rx 7900 xtx, as long a you don't need Cuda support or tensor cores

−1