Viewing a single comment thread. View all comments

DavesEmployee t1_j1426ms wrote

Reply to comment by caninerosie in [D] Running large language models on a home PC? by Zondartul

4090s don’t support NVLink unfortunately 🥲

2

BelialSirchade t1_j174fni wrote

You don’t need NVLink though, PyTorch support model parallelism through deepspeed anyways, so go ahead and buy that extra 4090

2

caninerosie t1_j14738h wrote

really? NVIDIA is so weird

1

DavesEmployee t1_j147fki wrote

I think it’s because they’re mostly used for games which almost never take advantage of the technology. You can tell from the designs that they were going to support it but the feature was taken out probably due to price or power concerns

2