csreid t1_iwlyxnz wrote
>Also, I can request up to 372GB VRAM, is there any large language model (#parameters > 100B) that I can actually download and run "locally"?
I've never done anything non-trivial with LLMs but even using 32 bit floats for 100B parameters should take 400 gigs of RAM, right?
SJ5125 t1_iwwfez9 wrote
Most would use bfloat16 for LLMs
Viewing a single comment thread. View all comments