Enterprise RAG Architecture for Scientific R&D: Retrieval, Evidence and Provenance

Enterprise RAG for scientific R&D should not be designed as “documents into a vector database, vector database into an LLM.” Scientific users need more than semantic similarity. They need evidence that can be inspected, permissions that survive retrieval, provenance that survives synthesis and evaluation that measures whether the answer is actually supported.

Start with a source model, not embeddings

Before chunking, define the identity of the source: document ID, version, section hierarchy, author or system of record, timestamps, project or study context, access policy and any structured entities that matter to the workflow.

Embeddings are one retrieval representation. They should not become the canonical representation of the source.

Parse for meaning and recoverability

Scientific documents often contain tables, figure captions, references, section dependencies and repeated terminology. Chunk boundaries should preserve enough hierarchy to reconstruct why a passage means what it means.

Store parent-child relationships so that a retrieval result can expand from a matched fragment to the surrounding section or source artifact without losing identity.

Authorization must participate in retrieval

Permission checks after retrieval are too late if unauthorized content already reached the model context. Filter candidate evidence using the user’s effective access rights before context construction.

In multi-project or multi-tenant environments, this may require a combination of metadata filters, row-level controls, index partitioning and service-layer authorization. The correct mechanism depends on scale and threat model; the requirement is that retrieval cannot weaken source-system permissions.

Use hybrid retrieval deliberately

Scientific language contains identifiers, exact terms and domain-specific phrases that lexical retrieval handles well, while semantic retrieval helps with conceptual similarity and paraphrase. A strong baseline often combines both, then reranks a candidate set using the actual query and document context.

query understanding
      ↓
metadata / permission filters
      ↓
lexical candidates + semantic candidates
      ↓
fusion
      ↓
reranking
      ↓
evidence assembly

Separate retrieval relevance from evidence sufficiency

A passage can be relevant without supporting the claim the user wants to make. Add an evidence-sufficiency step that asks whether the retrieved set contains enough information to answer, compare or conclude.

This is where abstention becomes a product feature. “I found related material but not enough evidence to support that conclusion” is often more useful than confident synthesis.

Build provenance at claim level

Instead of attaching three citations to the end of a generated paragraph, decompose the response into claims and map each claim to the evidence used to support it. That creates a review surface:

Claim A → Source 12 / section 4.2 / version 7
Claim B → Source 03 / table 8 / version 2
Claim C → insufficient evidence → flagged

Claim-level provenance is valuable for users and for evaluation because it lets the system measure support, not merely citation presence.

Evaluation needs a leakage-resistant design

Scientific RAG evaluation should separate the corpus used for retrieval from labels or historical outcomes that would reveal the answer. A benchmark can look excellent while being useless if the future answer, regulator question or review annotation is accidentally included in indexed context.

I would track at least retrieval recall, reranker quality, evidence support, answer completeness, abstention precision, citation correctness, latency and reviewer agreement.

Reference architecture

Sources
  ↓
versioned ingestion + parsing
  ↓
canonical source store
  ├── lexical index
  ├── vector index
  └── structured metadata
          ↓
user identity + policy
          ↓
query understanding
          ↓
permission-aware hybrid retrieval
          ↓
reranking + evidence assembly
          ↓
LLM synthesis with structured claims
          ↓
claim-to-source verification
          ↓
answer + provenance + abstention state
          ↓
human review + evaluation telemetry

What I would optimize only after quality is measurable

Once the evaluation harness is credible, then tune chunking, embedding models, rerankers, caching, context windows and model routing. Without a trustworthy measurement system, those changes are architecture by anecdote.

The goal of scientific RAG is not to retrieve text that sounds related. It is to construct an inspectable chain from question to evidence to claim.

That shifts RAG from a chatbot feature into an evidence architecture—much closer to what a serious scientific workflow needs.