You’re glossing over a lot of details here, which is where most of the pain is.
Properly chunking the data, handling non-standard text formatting in source documents, not even having OCR’d text in source documents, having disparate indexes available per client, minimizing hallucinations even with properly context data, and more.
It’s not too different from you yourself searching for a few documents and reading them before formulating an answer.