LetterRip t1_j3l42en wrote on January 9, 2023 at 8:30 AM

Reply to comment by learningmoreandmore in [D] I want to use GPT-J-6B for my story-writing project but I have a few questions about it. by learningmoreandmore

You can use the GPT-J-6B 8bit, and can do finetuning on a single GPU with 11 GB of VRAM.

https://huggingface.co/hivemind/gpt-j-6B-8bit

You could probably do a fine tune and test fairly cheaply using google colaboratory or colaboratory pro (9.99$/month).

learningmoreandmore OP t1_j3l4cu0 wrote on January 9, 2023 at 8:34 AM

Thanks! Would this be able to scale and handle more computations or is this only for personal use? I wonder why most people wouldn't be using this version if it's so efficient computation-wise

mr_birrd t1_j3l7rjf wrote on January 9, 2023 at 9:21 AM

Cause it is worse metric wise.

LetterRip t1_j3meu7o wrote on January 9, 2023 at 4:08 PM

Same license as the 32 bit version so commercial usage is fine (apache-2.0 - see the page for details) Should give similar results and scaling (according to the link above is 1-10% slower inference).