Mental BoundRAG addresses a core limitation of large language models: they only know what was in their training data. For domains that change frequently — support docs, internal knowledge bases, regulations, market data, your own product spec — pretrained knowledge alone is insufficient and often dangerously outdated. RAG fixes this by querying external sources first, then passing the retrieved context to the model as additional input alongside the user's question.
The typical pipeline has four stages. Ingest: documents are chunked into passages of a few hundred tokens, embedded into vectors, and stored in a vector database. Retrieve: the user query is embedded with the same model and used to search the index for the most semantically similar chunks. Re-rank: a smaller model or heuristic reorders the top candidates so the most relevant pieces sit at the top of the context window. Generate: the LLM receives the original question plus the retrieved passages and produces an answer grounded in that context, often with inline citations back to the source.
Each of those stages has tradeoffs. Chunking that splits sentences mid-thought breaks retrieval; chunking that's too coarse buries the answer in irrelevant text. Embedding model choice affects both cost and accuracy — domain-specific embeddings outperform general ones on technical content but require more setup. Retrieval-only systems return ranked passages without generating new prose, which is sometimes what you actually want when accuracy beats fluency.
When done well, RAG produces accurate, citeable answers without retraining the model. It is the right pattern when your data changes faster than you can fine-tune, when traceability and citations matter to your users (legal, healthcare, finance), and when scope is bounded enough that a small set of documents covers most queries. It is the wrong pattern when the underlying task needs the model to reason over data rather than recall it — that's where fine-tuning or an agent with tool use earns its keep.
For production deployments, the make-or-break work is evaluation. Build a labeled set of representative queries, measure retrieval precision and answer faithfulness separately, and version the entire pipeline so a tweak to chunk size doesn't silently regress your accuracy on real customer questions.