Submitted by Numerous_Talk7940 t3_11w9hkj in deeplearning
Hi there,
I want to upgrade my GPU since I get continuously more involved into deep learning and training model every day. The two choices for me are the 4080 and 4090 and I wonder how noticeable the differences between both cards actually are. That is, will the Training be 2x faster or just 1.2? What actually is the benefit of investing more money, it my budget is not capped.
For deeplearning higher VRAM is always preferable. 12/16GB limits the kind of models you can run/infer. With LLMs getting democratised it's better to be future proof.
12/16gb is already hitting limit of what's available on market for consumer gaming GPU. Only GPU for deeplearning with more than 16bgb is 4090 which is already out of range for most individual at $1500
[deleted]
Yep you can get a 3090 for 700
This might help:
https://timdettmers.com/2023/01/16/which-gpu-for-deep-learning/
I am not confident comparing these charts perfectly, it 'looks like' the 4090 is superior to the 3090 (3090 because it has the same VRAM, which is an important factor as I have learned), but by how much? Does the price justify the advantages of the 4090?
No. I'd get a used 3090. Save the rest of your money for when you have more experience and a better grasp of the kinds of problems you'd like to solve and the hardware needed to run the corresponding models. Then you'll realize either that your hardware is sufficient as is (with a 3090), a 4090 would actually benefit you, a card with 48GB of VRAM is essential (ie. You need either an Ada Titan if it comes out or 2x 3090s), or it's way too expensive to run on consumer hardware and just use cloud GPU instances with A100s or H100s instead. But the 3090 will be a great card for now, and a used one in great condition (sometimes even open box or with an active warranty) can be found easily on the hardwareswap subreddit for $800 or even less.
I see and took a look, the 3090 ti is sometimes available for only 100 euros more (used that is). Is that something I could consider aswell?
Yes that's likely better actually since its a much newer card (less likely to have been mined on) just make sure you look into tuning it for efficiency as the card is designed to run at 450W (which is insane). I can't direct you to any guides for the 3090Ti specifically but I'd just google 3090Ti undervolt. The 3090 should probably also be undervolted too, you really don't need these cards to be hitting their power limits of 450W and 350W respectively, tuning them to a more reasonable 325-350W and 280-300W makes way more sense.
Although since you are in Europe, what does pricing look like roughly for used 3090s/3090Tis versus new 4090s?
3090 900-1000, 3090ti 1200-1300 4090 new 1750, which is why I considered still a new 4090, because the price gaps between those 3 Options seem odd.
I'd say in that case just go new 4090 unless you can get a used 3090 for half that or a used 3090Ti for a little over half. I'm surprised that the difference is that small, I guess I'm accustomed to the used market in the US which I imagine is quite a bit larger
I see, thanks for your help! The 4090 is from palit though while the used 3090/3090ti are from MSI or Gigabyte, which are more expensive in general. For my use case however, the brand should not matter. Does the advice for undervolting still hold for the 4090?
all comes down to VRAM.
The speed is not the main difference you're going to notice, it's the VRAM. VRAM is a hard limit to work around, so it simply depends whether you need the extra.
Probably need an answer from someone who has both and has benchmarked some examples. (EDIT: and I do not!) Personally I find a lot of “law of diminishing(Edit) returns” with this type of thing. Also for me, since I spend 100x more time coding and testing will dummy sets… the actual speed of run is not as critical as people would expect…
> "law of diminishing returns"
FIXED
Ta!
Palit is a good brand from what I've heard! And yeah I'm not as familiar with it but googling the same thing regarding power limiting/undervolting the 4090 will bring up similar results about optimizing efficiency for it. You can likely get it not exceed 350W under load and likely even get a performance boost, not even regression. The 4090 is incredibly efficient given its performance, it also draws way less power when idling than the 3090 or 3090Ti. Had no idea the price gap was that small haha otherwise I would've recommended the 4090 straight away especially given the price in energy increase you've more than likely experienced
MisterManuscript t1_jcxo0zv wrote
You don't need the 40 series, they're designed with providing ML solutions to games. You're paying extra just to have an in-built optical-flow accelerator that you're not gonna use for model training.
The optical flow accelerator is meant for computing dense optical flow fields as part of many inputs to the DLSS feature that most new games use.
You're better off with the 30 series or lesser.