Detect Flaky GitHub Actions Jobs with OpenAI + Slack

Detect flaky GitHub Actions jobs—and get Slack reports with OpenAI insights

This n8n workflow automatically analyzes your GitHub Actions re-runs to spot flaky jobs, summarizes likely intermittent causes with OpenAI, posts a clear update to Slack, and files GitHub issues for confirmed flakes—while tracking flake rates in an n8n Data Table.

What this workflow does

• Runs daily (or on demand) to start the analysis for a configured repository, workflow, and branch.
• Fetches recent GitHub Actions runs and filters to the target workflow’s successful and failed attempts, including earlier runs when a job was re-run.
• Retrieves all jobs per run attempt (paging through results) to ensure matrix workflows are fully captured.
• Classifies job behavior by aggregating outcomes by job name: flaky, confirmed flaky (failed then passed via re-run on the same commit), broken, or one-off.
• Computes flake rates and extracts common failing step names.
• Compares against history stored in the flaky_jobs n8n Data Table, reporting only new or significantly worse flaky jobs.
• Uses OpenAI to group reported flaky jobs by likely intermittent cause based on failing step names.
• Posts a formatted report to Slack and creates GitHub issues (up to a configured limit) for confirmed flakes.
• Upserts verdicts and latest flake rates back into the flaky_jobs Data Table.

Use cases

• SaaS ops teams want automated visibility into flaky CI/CD behavior that wastes engineering time.
• Automation engineers need a repeatable workflow to turn flaky test signals into actionable GitHub issues.
• Teams running GitHub Actions matrices want comprehensive job coverage and trend-based detection.

Technical details

• Integrations: GitHub, Slack, OpenAI
• Node categories used: if, set, code, no op, slack, github
• Storage: n8n Data Table named flaky_jobs for flake rates and verdicts
• Auth: HTTP Header Auth credential (setup step referenced in the source)