Give your LLM access to private data by injecting documents into the prompt.
The Knowledge Cutoff
LLM
Generic knowledge only
Your Company Data
Private, internal docs
The LLM has never seen your private documents.
"What were Q3 sales?" → "I don't have access to your company data..."
Retrieval-Augmented Generation
Find relevant documents from your data store
Inject retrieved context into the prompt
LLM produces a grounded, sourced answer
Step 1: Retrieve
Search your document store for relevant matches
Q3 Report.pdf
MATCHSales Data.csv
MATCHQ1 Report.pdf
HR Policy.pdf
Meeting Notes
MATCHProduct Spec
Full Document
~4000 tokens
Split documents into manageable chunks so retrieval is precise
Vector Search
Convert question to a vector, find nearest document chunks
Nearest chunks are the most relevant matches
Without Context
With Context (Augmented)
Token Budget Problem
8K context window — everything must fit
Balanced Budget
LLM generates a grounded answer from the augmented prompt
"Your Q3 revenue was $4.2M, up 18% YoY, primarily driven by enterprise contracts."
"Revenue was approximately $3M based on typical industry averages..."
"$4.2M, up 18% YoY, driven by enterprise contracts."
[Source: Q3 Report]
The Full RAG Pipeline
Data flows left → right through the pipeline
When RAG Isn't Enough
RAG Can Answer
RAG Cannot Act