Retrieval-augmented generation gets discussed as a model technique. In practice, when one of these systems underperforms, the model is almost never the binding constraint. The retrieval is.
If the right passage is not in the context, no model recovers it. If five nearly-identical passages are, the answer gets muddier, not sharper.
The parts that decide quality
Chunking. Splitting on a fixed token count cuts sentences off from the thing that makes them meaningful. Split on structure — sections, functions, headings — and the retrieved unit is a unit of meaning.
What you embed versus what you return. Embedding a summary and returning the full passage often beats doing both on the same text.
Recency and authority. Two documents can both be relevant and one can be obsolete. Pure similarity has no opinion about that. Your ranking must.
Evaluation on retrieval alone. Before judging answers, measure whether the correct source made it into the context at all. Most disappointing systems fail there, and no amount of prompt work fixes it.
The unglamorous conclusion
The work is librarianship — deciding what is worth keeping, how it is organised, and what is authoritative. That was true of search before any of this, and it stayed true.