What are the costs for all the services? I assume GPT-4 is billed per request and Pinecone per hour?

You could use faiss instead of pinecone and alpaca instead of gpt-4

True, but im not sure how much cheaper that would really be.

Instead of FAISS, you can also use a truly vector search database like Qdrant. It's open-source and also offers a generous free tier offering in the cloud https://qdrant.tech

Can it have good performance as openai’s GPT?

Will you add semantic chunking?

GPT charges per token so it depends on the length of the document

It will have better performance relative to the knowledge in the documents. It's the comparison of GPT-4 with global knowledge vs GPT-4 with local knowledge.

It *is* OpenAI's GPT. Through an API.

Can you give me a little bit more info on this ? I'm interested but also very slow

Sure:

1. Suppose you had 1 million embeddings of sentences, and one vector you want the closest sentence to. If the vectors were a single number, you could just do a binary search, and you'd be done. If they are higher dimensionality, it's a lot more involved. Pinecone is a paid product doing this. Faiss is a library by facebook, which is very good too, but is free.
2. Recently, Facebook released the LLama models. They are large language models. ChatGPT is also a LLM, but after pretraining on a text corpus, you train it with human instructions, which is costly and time-consuming. Stanford took the LLama models, and trained them with ChatGPT. The result is pretty good not AS good, but pretty good. They called it "Alpaca".

So If I had a pdf, I could use faiss to make am it into an embedding, and then llama / alpaca to use the pdf as a base for a chatbot ?

I heard of people talking about using ANNOY for approximate nearest neighbor search. How is ANNOY compared to pinecone and faiss? Are pinecone and faiss self-hostable?

Idk, I've never heard of it.

could there be something like this, but for code?

I just found this a few days ago and actually used it as a prototype base to learn the ropes, so thanks op! I ended up gutting the ingest flow in favor of an additional upload api route to make it more flexible, but overall it was a good example/guide to follow. Nice work.