Automate Markdown Reindexing with Supabase & Webhooks

Streamline Your Markdown Documentation with Automated Reindexing

Stay ahead of the curve with our 'Automate Markdown Reindexing with Supabase & Webhooks' workflow. Designed for n8n users, automation engineers, and SaaS operators, this workflow automates the reindexing of your Markdown documentation directly into your Supabase Postgres pgvector table. Seamlessly embed and update your documentation without lifting a finger.

What this workflow does

• Operates on a set schedule or triggers instantly when a POST request is received on the webhook endpoint.
• Fetches Markdown source documents from a predefined HTTP API endpoint, perfect for FAQs, blog posts, or any text-heavy content.
• Processes the content by removing frontmatter, dividing it into H2-based sections, further chunking long sections with intelligent overlap, and batching these chunks for optimal embedding.
• Calls a Supabase Edge Function to create precise and accurate embeddings for each batch of text chunks.
• Upserts the processed chunk’s source, index, content, and pgvector embedding into your Supabase Postgres `rag_chunks` table, effectively handling duplicates with conflict updates.
• Finalizes the process by cleaning up outdated chunks, ensuring only the most recent and relevant data is retained in your database.

Use cases

• Enhance search functionality: Keep your Markdown-based knowledge base updated effortlessly, making sure search queries return accurate and relevant results.
• Automate content management: Perfect for automated documentation updates in SaaS applications, ensuring users receive the latest information.
• Optimize database performance: Regularly clean and update your vector table, maintaining database integrity and performance.

Technical details

• Nodes and Services: Leverages n8n's webhook node, HTTP request node, code node, split in batches node, and sticky note node for effective processing.
• Infrastructure: Utilizes Supabase Edge Function for embedding creation and Supabase Postgres for vector storage.
• Setup Requirements: Configure HTTP Header Auth for source API and Supabase Edge Function calls, and ensure Supabase Postgres credentials are in place.
• Database Schema: Requires a `public.rag_chunks` table with a pgvector `embedding` column and a suitable primary key (source, chunk_idx).

Experience the power of automated Markdown reindexing—where accuracy meets efficiency. Get started today and elevate your content management with minimal effort and maximum impact.