n8n Sub-Workflow: Fetch & Cache Web Pages with Bright Data

Fetch, render, and cache web pages with Bright Data—fast and consistent

This n8n sub-workflow pulls a URL using Bright Data Web Unlocker, serves results from an n8n Data Table cache when fresh, and returns a standardized response object containing status, content, and detailed error information.

What this workflow does

• Receives requests from other n8n workflows as a sub-workflow, including a URL plus optional output format and max-age settings.
• Validates and normalizes inputs: checks the URL, normalizes the requested format, and computes a cache freshness window.
• Checks the n8n Data Table cache for an exact URL match and returns cached content immediately when it is still fresh.
• On cache miss, fetches via Bright Data and inspects the response body for Bright Data error envelopes, empty payloads, or unusually thin content.
• Retries with browser rendering when needed: if the first fetch looks empty or too thin, it re-requests the page with browser rendering enabled to better handle JavaScript-heavy pages.
• Returns a consistent result including ok, from_cache, rendered, bytes, content, and error.
• Upserts the latest response into the Data Table for faster future calls.

Use cases

• Building reliable web scraping pipelines where repeated URL requests must be deduplicated and accelerated via caching.
• Scraping modern, JavaScript-rendered pages that may require Bright Data rendering after an empty/thin first response.
• Creating SaaS workflows that need consistent outputs (status/content/error) from a shared sub-workflow.
• Monitoring or QA workflows that track whether data came from cache vs. live fetch using from_cache and rendered.

Technical details

• Integrations: Bright Data Web Unlocker (via HTTP Header Auth credential in n8n).
• Cache layer: n8n Data Table named scrape_cache (with columns like url, status, content, fetched_at).
• Workflow building blocks: if, set, code, no op, data table, and a sticky note.
• Config: update default unlocker zone name and default values for format and max_age_minutes in the workflow settings.