Build a verifiable question set
Before choosing a database, collect real questions and expected source documents. Define which passage supports each answer and when the system must abstain. This becomes an acceptance set instead of relying on a few impressive conversations.
Document preparation sets the quality ceiling
Scans, tables, diagrams and policies need different extraction paths. Validate OCR, structure, encoding, duplicates and freshness. Chunk size should follow document structure, while metadata preserves section, date, owner and access level.
Measure retrieval separately from generation
If the right source is not retrieved, inspect indexing, metadata and query construction. If the source is present but the answer is wrong, inspect prompt, context and model behavior. Hybrid semantic and lexical search often helps with identifiers and exact terms.
Enforce current permissions at query time
Roles and document status change. Retrieval must evaluate current authorization and prevent restricted content from leaking through citations, suggestions or logs.
Operate the index like a production system
Use update queues, error handling, retries, schema versions and rollback. Monitor freshness, latency, concurrent load and failed components. A useful RAG system is measurable, explainable and maintainable.