Use byteandbit to load it with 8 bit and then you can run it on a local machine. Basically I think running the model is not a problem, but am not very optimistic about the quality of the generated story. I am using gpt3 to generating ad copies and to be honest it still takes a lot of efforts to tune the prompts to get some decent results, let along using the open-sourced model.
Human_Ad8482 t1_j3lpxgx wrote
Reply to [D] I want to use GPT-J-6B for my story-writing project but I have a few questions about it. by learningmoreandmore
Use byteandbit to load it with 8 bit and then you can run it on a local machine. Basically I think running the model is not a problem, but am not very optimistic about the quality of the generated story. I am using gpt3 to generating ad copies and to be honest it still takes a lot of efforts to tune the prompts to get some decent results, let along using the open-sourced model.