Viewing a single comment thread. View all comments

crayphor t1_jawrahh wrote

Reply to comment by DaTaha in [P] LazyShell - GPT based autocomplete for zsh by rumovoice

You can use a smaller model like GPT-2. You are not going to get ChatGPT performance without a terabyte of VRAM, but if you want to try something locally, GPT-2 exists.

2

Art10001 t1_jayfgsc wrote

I suggest another model such as OPT or even Flan-T5, because they're much easier to setup than OAI's outdated instructions that use outdated package versions that effectively demand a for-purpose VM or Docker.

3

Pat3418 t1_jbjmcty wrote

Have you done something like this? Any references you can share? Would love to have an offline tool like this…

1

Art10001 t1_jbjv13z wrote

I once setup the smallest model of Pygmalion to act as a chatbot. The UI was premade and Gradio-based.

It worked well. I had tried Flan-T5 first, but the UI did not recognize the model kind (yet.)

1