{"product_id":"extract-website-markdown-chunks-with-apify-http-request","title":"Extract Website Markdown Chunks with Apify \u0026 HTTP Request","description":"\u003ch3\u003eExtract public website passages into clean Markdown chunks using Apify \u0026amp; n8n\u003c\/h3\u003e\n\u003cp\u003eThis n8n workflow runs Mako’s \u003cstrong\u003eWeb Content Crawler on Apify\u003c\/strong\u003e to fetch website content, validate coverage, and return \u003cstrong\u003eMarkdown chunks\u003c\/strong\u003e ready for account research or retrieval workflows—each chunk includes its source URL, heading path, content hash, and crawl timestamp.\u003c\/p\u003e\n\n\u003ch3\u003eWhat this workflow does\u003c\/h3\u003e\n\u003cul\u003e\n  \u003cli\u003e\n\u003cstrong\u003eManual execution with explicit start URLs:\u003c\/strong\u003e when you run it manually, it defines \u003cstrong\u003e1–10 start URLs\u003c\/strong\u003e and crawl limits.\u003c\/li\u003e\n  \u003cli\u003e\n\u003cstrong\u003eTriggers an Apify Actor run:\u003c\/strong\u003e it starts the actor \u003cstrong\u003eagency-shift\/web-content-crawler\u003c\/strong\u003e using the \u003cstrong\u003eApify REST API\u003c\/strong\u003e via n8n \u003cem\u003eHTTP Request\u003c\/em\u003e.\u003c\/li\u003e\n  \u003cli\u003e\n\u003cstrong\u003ePolls until completion (or deadline\/failure):\u003c\/strong\u003e the workflow polls the run status until it finishes, a polling deadline is reached, or a status request fails. It then attempts to abort the run on the stop path, with an additional \u003cstrong\u003e180-second Actor timeout\u003c\/strong\u003e applied.\u003c\/li\u003e\n  \u003cli\u003e\n\u003cstrong\u003eChecks RUN_SUMMARY for coverage \u0026amp; truncation:\u003c\/strong\u003e it fetches the \u003cstrong\u003eRUN_SUMMARY\u003c\/strong\u003e from Apify Key-Value Store and stops if the run failed, coverage is incomplete, or output is truncated\/empty.\u003c\/li\u003e\n  \u003cli\u003e\n\u003cstrong\u003eReturns one output item per chunk:\u003c\/strong\u003e it retrieves dataset items, validates each page’s Markdown\/chunk schema, and outputs chunk content plus:\n    \u003cul\u003e\n      \u003cli\u003e\n\u003cstrong\u003esourceUrl\/sectionUrl\u003c\/strong\u003e, \u003cstrong\u003eheading path\u003c\/strong\u003e\n\u003c\/li\u003e\n      \u003cli\u003e\n\u003cstrong\u003econtent hash\u003c\/strong\u003e and \u003cstrong\u003ecrawl time\u003c\/strong\u003e\n\u003c\/li\u003e\n      \u003cli\u003ean \u003cstrong\u003eoversized\u003c\/strong\u003e flag when applicable\u003c\/li\u003e\n    \u003c\/ul\u003e\n  \u003c\/li\u003e\n\u003c\/ul\u003e\n\n\u003ch3\u003eUse cases\u003c\/h3\u003e\n\u003cul\u003e\n  \u003cli\u003ePreparing public website passages for \u003cstrong\u003eaccount research\u003c\/strong\u003e and company knowledge bases\u003c\/li\u003e\n  \u003cli\u003eBuilding a \u003cstrong\u003eretrieval workflow\u003c\/strong\u003e (RAG indexing) from web pages split into Markdown chunks\u003c\/li\u003e\n  \u003cli\u003eQAing crawl coverage before connecting downstream processing\u003c\/li\u003e\n\u003c\/ul\u003e\n\n\u003ch3\u003eTechnical details\u003c\/h3\u003e\n\u003cul\u003e\n  \u003cli\u003e\n\u003cstrong\u003eApify REST API\u003c\/strong\u003e via n8n \u003cstrong\u003eHTTP Request\u003c\/strong\u003e (Actor: \u003cem\u003eagency-shift\/web-content-crawler\u003c\/em\u003e)\u003c\/li\u003e\n  \u003cli\u003e\n\u003cstrong\u003eAuth:\u003c\/strong\u003e set \u003ccode\u003eAuthorization: Bearer \u0026lt;YOUR_APIFY_TOKEN\u0026gt;\u003c\/code\u003e using an n8n HTTP Header Auth credential\u003c\/li\u003e\n  \u003cli\u003e\n\u003cstrong\u003eNodes used:\u003c\/strong\u003e \u003cem\u003eManual Trigger\u003c\/em\u003e, \u003cem\u003eif\u003c\/em\u003e, \u003cem\u003ecode\u003c\/em\u003e, \u003cem\u003ewait\u003c\/em\u003e, \u003cem\u003esticky note\u003c\/em\u003e, \u003cem\u003eHTTP Request\u003c\/em\u003e\n\u003c\/li\u003e\n\u003c\/ul\u003e","brand":"N8N Commerce","offers":[{"title":"Default Title","offer_id":46157940261043,"sku":"N8N-20339","price":42.99,"currency_code":"GBP","in_stock":true}],"thumbnail_url":"\/\/cdn.shopify.com\/s\/files\/1\/0749\/6279\/6723\/files\/hZRRFW3TWpLd8A196uDNW_Hdyh7RoB.png?v=1791019062","url":"https:\/\/buyflowscripts.com\/products\/extract-website-markdown-chunks-with-apify-http-request","provider":"N8N Commerce","version":"1.0","type":"link"}