Index New Google Drive Files to Pinecone with OpenAI Embeddings (n8n)

Index every new Google Drive file into Pinecone—automatically (with OpenAI embeddings) in n8n

This n8n workflow watches a specific Google Drive folder for newly created files, generates OpenAI embeddings from the file contents, and upserts the resulting vectors into your Pinecone index—while using an n8n Data Table to prevent duplicate indexing by logging each ingested file ID.

What this workflow does

• Runs every minute: A Google Drive Trigger checks the selected folder for newly created files.
• Deduplicates by file ID: Before processing, it checks an n8n Data Table to see whether the Google Drive file ID is already recorded as ingested.
• Downloads file content: New files are downloaded as binary data, then loaded into the workflow for text processing.
• Splits into overlapping chunks: The workflow splits the content into chunks (with overlap) and attaches metadata including file ID and file name.
• Generates OpenAI embeddings: Each chunk is embedded using an OpenAI embeddings model (e.g., text-embedding-3-small).
• Upserts into Pinecone: The chunk vectors plus metadata are inserted into the specified Pinecone index.
• Logs ingestion for future runs: It records file ID, file name, and an ingestion timestamp in the same n8n Data Table to avoid re-embedding.

Use cases

• Build a RAG-ready document index from files teams drop into a Google Drive folder.
• Automatically refresh Pinecone with new documents without manual ingestion jobs.
• Prevent costly re-embedding by tracking ingested Google Drive file IDs.

Technical details

• n8n workflow triggers: Google Drive Trigger (runs every minute on new files in a selected folder)
• Storage/deduping: n8n Data Table (e.g., rag_ingested_files)
• Embeddings: OpenAI embeddings (choose a model such as text-embedding-3-small)
• Vector database: Pinecone index (configured so the index dimension matches the embeddings model)
• Processing: Uses chunking with metadata attachment (file ID, file name) and inserts chunk vectors into Pinecone

Set up Google Drive OAuth2 credentials, OpenAI API credentials, your Pinecone index, and the n8n Data Table ID used for deduplication/logging—then let your Pinecone index stay continuously up to date.