Retrieval-augmented generation is the pattern that made enterprise AI assistants plausible: ground the model in your documents, and it answers from your knowledge instead of its imagination. The demo takes a weekend. The production system takes months — and the difference is almost entirely in the retrieval, not the generation.
Scale changes the problem
On ten documents, any retrieval works; the right passage is nearly always found. On ten thousand documents, near-duplicates, outdated versions, and conflicting policies flood the candidate set. The model then does exactly what it is designed to do — generate a fluent answer from whatever it was given — and fluency is precisely what makes a wrongly-grounded answer dangerous. The system fails quietly, with confidence.
The unglamorous work that decides quality
Production-grade grounding is document hygiene and information architecture. Which source is authoritative when two conflict? How is stale content retired from the index? How are tables, scans, and forms parsed so their meaning survives chunking? Who can see which documents — enforced at retrieval time, not in the prompt? These questions decide answer quality long before model choice does.
Measure grounding, not vibes
The teams that succeed treat retrieval as a measured system: a golden set of real questions with known correct sources, retrieval hit-rates tracked over time, citations required in every answer, and an escalation path when confidence is low. With that harness, quality problems become debuggable — was the right passage retrieved, ranked, and used? Without it, the only signal is a user who stopped trusting the answers, and that signal arrives too late.
