Slack DevOps Incident Triage with GPT-4.1, Prometheus & Jira

Turn Slack incident reports into Jira (or GitHub) tickets—with GPT-4.1, Prometheus, Loki, and human approval

This n8n workflow automates DevOps incident triage by listening to Slack alerts, using a GPT-4.1 agent to investigate via Prometheus metrics and Loki logs (plus RAG from past incidents), saving the analysis to PostgreSQL, and—only after explicit approval—creating a Jira or GitHub issue and triggering Terraform remediation.

What this workflow does

• Detects new incidents in Slack by triggering when a message is posted in your configured Slack incident-response channel.
• Normalizes the incident context (message text, channel, thread timestamp, reporter) and creates a session ID for thread-based memory.
• Performs AI-assisted investigation using a GPT-4.1 agent with tool-calling to query:
  • Prometheus for metrics evidence
  • Loki for relevant log findings
  • a vector-store RAG knowledge base for past incident patterns
• Generates a structured RCA (JSON) including root cause, confidence, evidence, remediation, risk level, and service.
• Saves the incident analysis to a PostgreSQL incidents table.
• Requests approval when remediation is proposed: if no concrete remediation is found, it posts an investigation update to Slack and stops; otherwise it posts an approval request and waits for a reviewer decision via a resume webhook.
• Closes the loop after approval: creates a ticket in Jira or GitHub Issues, triggers a Terraform Cloud run, re-queries Prometheus, adds a Grafana annotation, and posts a resolution summary back to the original Slack thread.
• Handles rejection safely by posting a rejection notice to Slack and stopping.

Use cases

• SaaS teams that triage incidents from Slack and want consistent, evidence-based RCAs.
• Automation engineers needing an n8n workflow that ties Slack → AI investigation → PostgreSQL → Jira/GitHub → Terraform → Grafana.
• DevOps incident response where changes must be approved by a human before remediation.

Technical details

• Trigger: Slack new message (incident-response channel)
• Nodes / integrations: if, set, slack, wait, jira, github
• AI: OpenAI GPT-4.1 agent with session memory and tool-calling
• Observability: Prometheus, Loki, Grafana annotation
• Data storage: PostgreSQL (incidents table)
• Remediation: Terraform Cloud run (post-approval)