Viewing a single comment thread. View all comments

edthewellendowed t1_jddoq57 wrote

Reply to comment by Icko_ in [P] Open-source GPT4 & LangChain Chatbot for large PDF docs by radi-cho

Can you give me a little bit more info on this ? I'm interested but also very slow

1

Icko_ t1_jdecnjx wrote

Sure:

  1. Suppose you had 1 million embeddings of sentences, and one vector you want the closest sentence to. If the vectors were a single number, you could just do a binary search, and you'd be done. If they are higher dimensionality, it's a lot more involved. Pinecone is a paid product doing this. Faiss is a library by facebook, which is very good too, but is free.
  2. Recently, Facebook released the LLama models. They are large language models. ChatGPT is also a LLM, but after pretraining on a text corpus, you train it with human instructions, which is costly and time-consuming. Stanford took the LLama models, and trained them with ChatGPT. The result is pretty good not AS good, but pretty good. They called it "Alpaca".
2

edthewellendowed t1_jdewxml wrote

So If I had a pdf, I could use faiss to make am it into an embedding, and then llama / alpaca to use the pdf as a base for a chatbot ?

1

saintshing t1_jdgwgt7 wrote

I heard of people talking about using ANNOY for approximate nearest neighbor search. How is ANNOY compared to pinecone and faiss? Are pinecone and faiss self-hostable?

1

Icko_ t1_jdh2pja wrote

Idk, I've never heard of it.

1