n8n Page Tools Workflow: Source-Linked Webpage JSON Dataset

Turn up to 10 public webpages into a source-linked JSON dataset—before you import anything

This n8n Page Tools workflow pulls webpage content using Circadian Page Tools on RapidAPI, converts the results into a structured research-sources.json dataset, and clearly separates successful source text from errors, empty pages, partial/truncated content, and unsupported response types—so your desk research and knowledge-base imports start clean.

What this workflow does

• Starts with your selected URL array (manual start). It checks the input format, rejects embedded URL logins, trims whitespace, and removes exact duplicate URL strings.
• Extracts each unique URL once via the fixed Page Tools extraction gateway on RapidAPI, using a 1-second batch interval and no automatic retries.
• Classifies every response into one of: extracted, source_http_error, partial_response, unsupported_content, empty_content, api_error, or invalid_response.
• Preserves diagnostic metadata for each record, including requested URL, final URL, API/source statuses, content type, and truncation flag.
• Builds one dataset containing a result summary, run-local source IDs, an assembly timestamp, explicit limitations, and error records with null Markdown so failures are visible.
• Outputs and downloads research-sources.json for review prior to research or knowledge-base import.

Use cases

• Pre-screen public URLs for knowledge-base ingestion (catch empty pages, partial HTML, or truncation).
• Rapid desk research: compile source-linked Markdown snapshots for review and citation.
• Compliance-friendly documentation: keep both successes and failures in one JSON export for traceability.

Technical details

• Circadian Page Tools via RapidAPI (Markdown extraction from webpages)
• n8n nodes: Manual Trigger, Set, Code, Sticky Note, HTTP Request, Convert to File

Important: The workflow does not crawl the links found within extracted pages; it only processes the provided URL set.