[D] Large Language Models feasible to run on 32GB RAM / 8 GB VRAM / 24GB VRAM Submitted by head_robotics t3_1172jrs 2 years ago in MachineLearning 51 comments 220
pyepyepie t1_j9bbg1b wrote 2 years ago Try to use both GPUs with this one: https://github.com/huggingface/accelerate https://huggingface.co/docs/accelerate/usage_guides/big_modeling https://huggingface.co/blog/accelerate-large-models Maybe it will help (the last link is clearer IMHO). Permalink 3
Viewing a single comment thread. View all comments