Fix Kubernetes CrashLoopBackOff with OpenAI, Slack & Sheets

Fix Kubernetes CrashLoopBackOff Incidents Automatically—Using OpenAI, Slack & Google Sheets

This n8n workflow detects Kubernetes pods stuck in CrashLoopBackOff, analyzes the failing logs with OpenAI, applies the most likely remediation via the Kubernetes API, and reports the incident to Slack, SendGrid email, and Google Sheets.

What this workflow does

• Triggers from Alertmanager or polling: It can receive a webhook from Prometheus Alertmanager or run on a 2-minute poll to list pods via the Kubernetes API.
• Filters only CrashLoopBackOff pods: It checks container status for CrashLoopBackOff and exits early with a “no action needed” webhook response when nothing is found.
• Collects evidence: For each failing pod, it retrieves the previous container logs and packages a log excerpt with pod metadata.
• Diagnoses with OpenAI: It uses OpenAI to classify the likely root cause, assign a confidence score, and select a recommended action. If the output is unparseable or below the confidence threshold, it escalates automatically.
• Remediates automatically: Depending on the recommendation, it can:
  • Delete the pod to restart
  • Patch deployment scale to increase replicas
  • Patch deployment template to roll back to an earlier ReplicaSet
  • Take no action when escalated
• Reports the outcome everywhere: It generates a short incident summary with OpenAI, posts to Slack, emails via SendGrid, appends a record to Google Sheets, and returns the result in the webhook response.

Use cases

• When Prometheus detects CrashLoopBackOff, automatically triage and remediate without waiting for a human.
• Reduce mean time to recovery by restarting or rolling back workloads based on failing logs.
• Maintain an auditable incident log in Google Sheets for ongoing reliability tracking.

Technical details

• Core workflow nodes: if, code, wait, merge, switch, webhook
• Integrations: Prometheus Alertmanager webhooks, Kubernetes API (via HTTP header auth using a service account bearer token), OpenAI, Slack, SendGrid email, Google Sheets