The source I found this post through also referenced Retrieval Augmented Generation (https://ai.facebook.com/blog/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models/) and it seems like they've integrated document selection into the back propagation of the model training. You couldn't do this with chat GPT but maybe smaller pretrained LLM that could be fine tuned on consumer hardware would be enough for just that part