Automate Price Scraping & Insights to Google Sheets with n8n

Automate Price Scraping & Insights to Google Sheets with n8n

Take control of your market analysis with our advanced n8n workflow, designed to automate the process of scraping product titles and prices from any target website and organizing the data into Google Sheets. Gain valuable insights every single day by leveraging cutting-edge technology, including OpenAI's GPT-4o-mini for market insights, right at your fingertips.

What this workflow does

• Runs daily at 9:00 AM using a schedule trigger.
• Fetches the target webpage HTML via HTTP and extracts product-related data blocks using CSS selectors.
• Parses the extracted data into individual product items by title and price, sorting them in descending order of price.
• Cleans price data into a numeric format, appending the current scrape date to each product record for accuracy.
• Appends these records to a Google Sheets tab, simultaneously calculating statistical price data — including minimum, maximum, and average prices.
• Generates market insights using OpenAI's GPT-4o-mini, providing a concise, plain-text summary of current market conditions.
• Converts the compiled product list into a CSV file, then emails the stats and AI insights with the CSV included via Gmail for comprehensive reporting.

Use cases

• E-commerce analysts seeking to streamline product price monitoring and comparison operations.
• Market researchers looking to automate daily data gathering and analysis for real-time decision making.
• Business owners aiming to track competitors' pricing strategies effortlessly and gain a competitive edge.
• Automation engineers in need of a sophisticated, yet manageable solution to integrate data science and AI insights into daily workflows.

Technical details

• Integration Nodes Used: HTML extraction, HTTP request, Code, Sort, Split Out, Gmail, Merge
• Technology Stack: n8n, Google Sheets, OpenAI (GPT-4o-mini), Gmail
• Setup Requirements: Update target URL and CSS selectors, add OpenAI credentials for GPT-4o-mini access, and configure Google Sheets and Gmail OAuth credentials.