Viewing a single comment thread. View all comments

Disastrous_Elk_6375 t1_jb8y5r2 wrote

GptNeoX should fit with 8bit and low prompt sizes. GptJ-7B should fit as well with 16bit inference. On smaller models you might even be able to do some finetuning for fun.

There's a couple of coding models from salesforce that you could fit comfortably. Check out FauxPilot for a copilot "clone".

8