Local Agentic RAG in n8n: Ollama + PGVector Document QA

Run a fully local Agentic RAG in n8n using Ollama + PGVector for document QA

This n8n template sets up an entirely local Agentic RAG (Retrieval Augmented Generation) system that can answer questions over your documents, dynamically selecting the right way to retrieve and analyze knowledge—without sending data to external AI services.

What this workflow does

Unlike standard RAG that mainly returns chunk-based matches, this template is designed to reason across your knowledge base and choose tools based on the question:

• Intelligent tool selection: switches between RAG lookups, full document retrieval, and SQL-backed querying when needed.
• Complete document context: retrieves entire documents instead of only chunk snippets when the question requires deeper grounding.
• Accurate numerical/tabular analysis: uses SQL for precise calculations—addressing common RAG weaknesses with spreadsheets and tables.
• Cross-document insights: connects related information across multiple documents to produce more coherent answers.
• Multi-file processing: handles multiple documents within a single workflow loop, supporting larger knowledge bases.
• Efficient storage for tables: stores tabular data using JSONB (without creating a new table per CSV), improving scalability.

Use cases

• Local internal documentation Q&A for SaaS ops and automation engineers who need accurate answers with full context.
• Spreadsheet/document QA where numerical accuracy matters (calculations, metrics, and tabular fields).
• Knowledge base support that combines insights across multiple files, not just isolated chunks.

Technical details

• Agentic RAG architecture designed for n8n with nodes such as: Set, Switch, Webhook, Postgres, Aggregate, Summarize.
• Ollama + PGVector for local embeddings and retrieval.
• Postgres for SQL-based numerical/tabular operations and structured storage (including JSONB for tabular data).

Author: Jadai kongolo (jadai_ai_automation). Start by running the table creation nodes first to set up your environment.