Nice way to get around the license problem.

Is Lora really associated with a quality loss? I thought it worked pretty well.

I think people don't know if they can legally do this under the llama license.

It's part of why the Lora approach to fine tuning is so nice, you don't have to share the original weights.

No, it's a non-commercial license focused on research cases.

Agreed. At least this work on top of llama is apache 2.0

I agree with you. Looking at papers like ToolFormer and so on, we are very close.

We are only a couple years away from AGI, which is what I've been saying for YEARS and getting yelled at here. The WaitButWhy article in 2016 was dead right

So you went through and signed it?

Nice. You signed it then?

You signed it?

Please everyone, go through to the link and sign it

Continuous retraining is something else.

I'll be training llama soon, I'll get back to you with how it goes.

Yeah, he doesn't get it. That's ok though, but to be wrong and be sure about it is a bummer

The fact that the fine tuning can improve instruction following is EXACTLY that. There's no distinction between predicting the next word, following instructions, or deep knowledge. They are all the same thing as far as an LLM.

This IS training. That's what it is. This is how "knowledge" got into the model in the first place

Okay, here's my friend turning the alpaca instructions into training data

https://github.com/lxe/llama-peft-tuner/blob/main/convert_alpaca_to_text.py

See how it's just turning it into a fat string?

False, they all do. The process of fine-tuning is identical to the initial pre-training, though perhaps with different settings. They're mostly setup to take q&a data for getting llama to take instructions better, but actually that's just text wrapped in some context and passed in straight up.

I was very confused by this as well but no, you can train new stuff.

I disagree that it's not viable to train a model. The problem is that the best public model (llama) is non-commercial.

That said, it's extremely possible to train things into it. There's a ton of new scripts floating around online. The Lora training is especially good.

The trouble with vectors is they are so limited. They're fine if you need to look up one distinct thing (and the vector gets the match right) but they're utterly useless if you'd like the model to learn about something in general.

Yes, I'm into it now. Code like this can be adapted to load bulk data instead of q&a.

I suspect some of the training parameters need to be adjusted a bit to prevent over fitting and obviously the data loading and templating needs to be removed.

https://github.com/lxe/llama-tune Or for a cooler approach where you make a Lora layer https://github.com/serp-ai/LLaMA-8bit-LoRA

I'm just going in deep. These are FUN

Yeah, like it's actually using a huge amount of brain power to figure out what the next word is. Just because that's how it works doesn't mean it's not intelligent.

If you want to be really good at figuring out what the next word is you have to be really smart

I do, still struggling with it

I'm aware of that part. The wording of the test that's injected is not public. If it was, if use it in my langchain scripts.

Again i really expect there's fine-tuning, we will see eventually maybe.

Based on how much langchain struggles to use tools and gets confused on them, I'd bet on fine tuning. I asked a contact to reveal what they're injecting into the prompt but it's not public information yet so i couldn't get it

By "in context learning" i take it you mean zero shot.

Yes, you can hot swap. Id be unsurprised if what Open-AI did is fine tune on how to use plugins in general by giving some examples combined with a little bit of zero-shot primer.

Something trained with ToolFormers technique and then told it can use a new, but similar, plugin is IMO going to generalize way better than something that's never used a plugin before.

My mistake, correct you are:

https://www.beren.io/2022-08-06-The-scale-of-the-brain-vs-machine-learning/

This post recons we using 800b parameters for language processing.

What's the underlying approach here? Just prompt engineering right?

I really really want to apply the ToolFormer paper to llama. They're both Facebook systems, you can get they've done it.

ToolFormer just seems like SUCH a good and thorough approach. There are quite a few gaps between the paper and building a working example, IMO, but it's clearly doable.

The way Facebook licensed the weights is frustrating me. We should all be passing around Alpaca trained, GPTQ quantized, SparseGpt optimized Llama derived models by now. Is there some telegram group i need to be in or something?