Viewing a single comment thread. View all comments

kross00 t1_jczd3i2 wrote

Reply to comment by Civil_Collection7267 in [D] Best ChatBot that can be run locally? by rustymonster2000

I’m having a hard time understanding what LoRA is and why it makes the 7B model better? I thought it only improves hardware requirements, but it also improves model coherency? This is all new for me

3

ericflo t1_jczqkmj wrote

LoRA is how you train llama into alpaca on consumer hardware

4

nolimyn t1_jd01nm3 wrote

the LoRA is like a modular refinement of the base language model, in this case it's the part that makes it feel like a chatbot / assistant, and makes it follow instructions.

you can see the same concept over at civitai.com, filter by LoRAs. Something like a LoRA for one character can be run on different checkpoints that focus on photorealism or anime, etc.

1