n8n FIFO LLM Queue: OpenAI Chat Completions + Postgres

Turn any OpenAI-compatible chat into a durable FIFO queue—built for n8n + Postgres

This n8n workflow acts as an n8n-first, durable FIFO gateway for a local OpenAI-compatible LLM. It accepts standard OpenAI Chat Completions requests, stores them in PostgreSQL, runs one inference at a time, and returns OpenAI-compatible JSON or SSE responses—so your automations never overwhelm your model.

What this workflow does

• Receives requests via a secured POST webhook at /v1/chat/completions (header authentication).
• Classifies incoming requests as either a new job submission or a control command (e.g., get result/wait, cancel, list queue).
• Queues jobs in PostgreSQL by storing submissions in a llm_queue_jobs table (created during the manual setup trigger).
• Processes the queue FIFO by locking and claiming the oldest eligible queued job under PostgreSQL locks.
• Calls the upstream chat completions URL, persists success results, or schedules a bounded retry back into PostgreSQL.
• Supports retrieval endpoints:
  • GET /v1/queue/tasks/:taskId to return durable status, queue position, and any stored error details for a specific task.
  • GET /v1/models to return configured comma-separated model identifiers without calling the upstream LLM.
• Runs every 5 seconds to recover expired leases and continue processing.
• Handles streaming by framing responses as OpenAI SSE when stream: true is requested.

Use cases

• Prevent concurrency spikes by funneling multiple n8n agents through a single-file FIFO LLM queue.
• Implement task-based retrieval so external systems can poll /v1/queue/tasks/:taskId for completion.
• Use OpenAI-compatible endpoints to integrate local LLMs with existing tooling that expects /v1/chat/completions.
• Run multi-tenant SaaS automation safely by queueing requests in PostgreSQL with durable state.

Technical details

• n8n webhook endpoints: POST /v1/chat/completions, GET /v1/queue/tasks/:taskId, GET /v1/models
• Secured via header authentication
• Nodes used: if, set, code, no op, switch, webhook
• PostgreSQL queue table: llm_queue_jobs with indexes created via manual setup