AnothaUselessComment t1_j9c9er6 wrote on February 20, 2023 at 9:34 PM Reply to [D] Large Language Models feasible to run on 32GB RAM / 8 GB VRAM / 24GB VRAM by head_robotics Yikes, this may be tough. I know you can try Bloom (like this blog post tried) and let it try and download overnight, but you may run into problems. (I've heard the download takes forever) https://enjoymachinelearning.com/blog/gpt-3-vs-bloom/ Though I will say, it's probably worth whatever cost you're trying to dodge just to hit an API, even if your hardware is great. Permalink 2
AnothaUselessComment t1_j9c9er6 wrote
Reply to [D] Large Language Models feasible to run on 32GB RAM / 8 GB VRAM / 24GB VRAM by head_robotics
Yikes, this may be tough.
I know you can try Bloom (like this blog post tried) and let it try and download overnight, but you may run into problems. (I've heard the download takes forever)
https://enjoymachinelearning.com/blog/gpt-3-vs-bloom/
Though I will say, it's probably worth whatever cost you're trying to dodge just to hit an API, even if your hardware is great.