Case Study: Enterprise RAG Knowledge Platform

01 Executive Overview

The Enterprise RAG Knowledge Platform is an end-to-end Retrieval-Augmented Generation (RAG) backend engineered to solve institutional knowledge fragmentation. In large enterprises, critical operational guidelines, compliance specifications, and technical manuals are scattered across unstructured PDFs, Markdown documents, and DOCX files.

Rather than relying on naive vector-similarity search (which frequently fails on exact acronyms, numbers, and part codes) or basic conversational wrappers, this system introduces an enterprise-grade retrieval pipeline combining dense semantic search (via PostgreSQL and pgvector) with sparse lexical BM25 retrieval, reciprocal rank fusion, cross-encoder reranking, and deterministic citation validation.

✓ Author Contribution & Authorship Boundary

Sole author and architect of the open repository. Designed the modular FastAPI microservice architecture, configured PostgreSQL schema migrations with pgvector, implemented the Reciprocal Rank Fusion (RRF) algorithm, authored automated integration test suites with pytest, and containerized the service using Docker and Docker Compose.

02 Problem & System Objectives

The Enterprise Knowledge Challenge

Enterprise adoption of Large Language Models (LLMs) faces three catastrophic failure modes when applied to private institutional data:

Hallucination & Ungrounded Answers: Standard LLMs generate authoritative-sounding but fabricated answers when domain context is missing or ambiguous.
Keyword vs. Semantic Retrieval Mismatches: Vector-only search often misses exact technical identifiers (e.g., error codes like ERR-0x892B, policy numbers, or SKU tags), while traditional keyword search misses semantic paraphrasing.
Cross-Department Data Leakage: Naive vector stores lack multi-tenant access boundaries, risking unauthorized disclosure of internal HR or executive data to unauthorized employees.

Target System Requirements

Sub-second Hybrid Retrieval: Merge dense semantic embeddings and sparse keyword scores to guarantee both high recall and high precision.
Deterministic Attribution: Enforce mandatory inline bracketed citations linking every asserted fact directly to source document chunks.
Strict Multi-Tenant Isolation: Enforce tenant, department, and role-based filtering directly at the database query layer prior to retrieval.
Containerized Zero-Drift Deployment: Package backend services and dependencies in self-contained Docker environments with automated CI quality gates.

03 Solution Architecture

The system follows a decoupled microservice architecture separating document ingestion, hybrid query execution, reranking, and generation.

System Architecture Flow

[Unstructured Sources: PDF, DOCX, MD, TXT]
                    │
                    ▼
       [Ingestion & Chunking Pipeline]
     (Hierarchical Splitting + Token Clamping)
                    │
       ┌────────────┴────────────┐
       ▼                         ▼
[Dense Embedding Engine]   [Sparse BM25 Indexer]
(OpenAI text-embedding-3)    (Lexical Inverted Index)
       │                         │
       ▼                         ▼
 [PostgreSQL + pgvector]   [Local Sparse Store]
 └─────────────┬─────────────────────────┘
               │  Multi-Tenant RBAC Filter (TenantID + DeptID)
               ▼
      [User Query Input]
               │
       ┌───────┴───────┐
       ▼               ▼
[Dense Top-K]    [Sparse Top-K]
       └───────┬───────┘
               ▼
  [Reciprocal Rank Fusion (RRF)]
  Score = Σ (1 / (k + rank_i))  where k = 60
               │
               ▼
   [Cross-Encoder Reranker]
   (ms-marco-MiniLM-L-6-v2)
               │
               ▼
  [Prompt Assembly & Context Clamping]
               │
               ▼
     [LLM Generation: GPT-4o]
               │
               ▼
 [Citation Verification & Guardrails]
               │
               ▼
 [Validated JSON API Response with Citations]

Component Rationale

PostgreSQL with pgvector: Eliminates the operational complexity of maintaining a separate dedicated vector database (e.g. Pinecone/Weaviate), allowing transactional document metadata and vector embeddings to share ACID guarantees and relational foreign keys.
Reciprocal Rank Fusion (RRF): Normalizes disparate score distributions between vector cosine similarities and unbounded BM25 scores without requiring manual weight calibrations.
Cross-Encoder Reranking: Performs joint attention across query-document pairs, eliminating false positives from initial candidate retrieval stages.

04 Implementation Details

Implemented Features

FastAPI Async Microservice: High-performance async REST API providing endpoints for document ingestion (POST /api/v1/documents/upload), query generation (POST /api/v1/chat/query), and system health (GET /health).
Tenant-Scoped Query Execution: Every SQL retrieval query applies strict WHERE clauses: tenant_id = :tenant_id AND access_level <= :user_level, guaranteeing tenant isolation before vector distance computations.
Deterministic Citation Extraction: Post-processes LLM generations to ensure every cited index [chunk_id] matches an existing retrieved passage in the prompt context. If ungrounded assertions are detected, the response flags a verification warning.

💡 Key Engineering Decision: Reciprocal Rank Fusion (RRF)

Cosine similarity scores from dense embeddings and BM25 scores from keyword indices have non-comparable scale and variance. Rather than linear weighted summation (which requires brittle heuristic hyperparameter tuning for every dataset), RRF uses reciprocal rank ordering: RRF_Score(d) = Σ [1 / (60 + rank(d))]. This provides robust, stable ranking across diverse document lengths and technical vocabularies.

05 Evaluation & Verification Evidence

The system has been evaluated through reproducible automated test suites and benchmark query comparisons:

Automated Test Suite (pytest): Tests cover vector indexing, BM25 scoring edge cases, reciprocal rank merging, Pydantic request validation, and API authentication.
Sub-second Latency: In local containerized testing, hybrid candidate retrieval across 1,000 embedded chunks executes in under 250 milliseconds prior to LLM generation.
CI Quality Gates: Automated GitHub Actions workflow executing code formatting (black, isort), static linting (flake8), and test runs on every pull request.

⚠️ Evaluation Scope Note

In accordance with portfolio credibility standards: This system is classified as a production-grade prototype. Latency and retrieval metrics were gathered in controlled developer test environments. Large-scale enterprise benchmarks across hundreds of thousands of concurrent users are subject to cloud hardware scaling.

06 Challenges & Engineering Trade-offs

Design Decision	Trade-off Accepted	Mitigation Strategy
PostgreSQL + pgvector	Slightly lower indexing throughput compared to pure in-memory vector databases.	Created HNSW vector indices (`m=16, ef_construction=64`) for sub-linear query scaling while maintaining transactional integrity.
Cross-Encoder Reranker	Adds 50–150ms computational overhead per query request.	Limited reranking input to top 15 candidate chunks rather than all retrieved candidates.
Local BM25 Store	Requires memory cache synchronization across distributed worker replicas.	In future cloud deployments, delegate BM25 sparse search directly to PostgreSQL full-text search (tsvector).

07 Future Roadmap

Local LLM & Embedding Engine Support: Integrate vLLM / Ollama backends to allow air-gapped on-premise execution without third-party API dependencies.
PostgreSQL Native Full-Text Search: Consolidate sparse BM25 indexing into native PostgreSQL tsvector columns to simplify multi-node horizontal scaling.
GraphRAG Knowledge Graph Integration: Incorporate entity-relationship graphs for complex multi-hop regulatory reasoning.

Enterprise RAG Knowledge Intelligence Platform