Benchmark RAG Chunking Strategies with Gemini, Drive PDFs

Benchmark RAG Chunking Strategies on Your Own PDFs—Automatically

This n8n workflow benchmarks three Retrieval-Augmented Generation (RAG) chunking approaches on your PDFs—fixed-size, semantic, and parent-document chunking—using free Gemini models, a strict LLM judge, and an emailed comparison report.

What this workflow does

• 1) Diagnostics setup in Google Drive: Drop PDFs into a Google Drive folder, then run the workflow to start the benchmark.
• 2) Parallel chunking experiments: The workflow creates three chunking “arms” in parallel:
  • Fixed-size recursive chunking with overlap (baseline)
  • Semantic chunking that cuts based on meaning shifts using Gemini embeddings
  • Parent-document chunking that pairs small precise child chunks with their parent context
• 3) Retrieval + answer generation: For each test question, the workflow embeds the question with Gemini (free embedding model), finds top matches using cosine similarity (implemented in plain Code for transparency), and generates answers from retrieved context using Gemini.
• 4) Strict evaluation: An LLM judge scores each strategy for:
  • Groundedness (are claims supported by the context?)
  • Recall (do expected keywords appear?)
• 5) Results + emailed report: Scores are merged into a comparison sheet, the winning chunking strategy is crowned by combined score, and you receive an email with per-question detail.

Use cases

• Find the best RAG chunking strategy for your document set before deploying to production.
• Evaluate semantic vs fixed chunking for different question types on your own PDFs.
• Tune RAG retrieval quality for a SaaS knowledge base without guessing.

Technical details

• Gemini API key setup: Use a free key from aistudio.google.com (free tier: 1,500 requests/day; no credit card).
• n8n nodes used: set, code, gmail, merge, sticky note, google drive.
• Configuration: Provide Google Drive folder ID, Gemini API key, and your email in the Workflow Configuration node; replace the sample questions in the Test Question Set node with your real user queries.