Redis Priority Job Queue for n8n Workers + Anthropic LLM

Redis Priority Job Queue for n8n Workers + Anthropic LLM — lane-based scheduling with Redis, worker wakeups, and HTTP/webhook control

Turn your n8n into a reliable distributed task runner: enqueue priority jobs into Redis, wake workers when work arrives, and (optionally) execute Anthropic LLM jobs or outbound HTTPS calls—complete with idempotency, TTL storage, leases, and webhook-based status/cancel endpoints.

What this workflow does

• Enqueue jobs via webhook: Accepts a POST to /webhook/queue-enqueue, validates job type, priority, payload size, delay, and attempts, then creates a job record with a token for deduping/fencing.
• Idempotent submission: Checks Redis for an existing job:{jobId} record and returns a duplicate response if the job already exists.
• Priority lanes + delayed scheduling: Stores jobs in Redis with a TTL, either pushing delayed work to queue:scheduled or routing immediate jobs into one of queue:critical, queue:high, queue:normal, or queue:low.
• Worker wakeups and polling: Wakes workers using Redis Trigger notifications or a 10-second schedule, then polls lanes in priority order with an anti-starvation rotation every N polls.
• Atomic claiming + execution: Loads and validates the job record using the message token, claims it with a lease, registers it in queue:processing, then executes:
  • llm: calls the Anthropic Messages API
  • http: performs an outbound HTTPS request
  • noop: runs a no-op echo handler
• Job lifecycle endpoints: Includes webhook endpoints to check queue/job status and cancel queued or delayed jobs.

Use cases

• Queue priority background tasks across multiple n8n workers without losing ordering guarantees.
• Run Anthropic LLM generations asynchronously with controlled retries/attempts and leasing.
• Safely manage delayed work (scheduling + cancellation) from external systems via webhooks.

Technical details

• Redis: job records (job:{jobId}), lane lists (queue:critical/queue:high/queue:normal/queue:low), scheduled queue (queue:scheduled), and processing registration (queue:processing), plus pub/sub notifications.
• n8n nodes: webhook, if, set, code, redis, switch, redis-based triggers/scheduling.
• Integrations: Anthropic Messages API (for llm) and outbound HTTPS calls (for http).