Grade LLM Answers & Route Low-Confidence with Judgment API

Grade LLM Answers Automatically & Route Low-Confidence Results for Human Review

This n8n workflow grades multiple AI answers against a reference solution using the Judgment API, computes a trust verdict, and automatically routes trusted outputs versus needs review items for human follow-up.

What this workflow does

• Runs on demand when you click Test Workflow.
• Loads a sample set of three question/answer/reference triples (or you can replace it with your own answer source).
• Sends each item to the community node n8n-nodes-judgment using the Grade Answers action to evaluate correctness, completeness, groundedness, relevance, clarity, and an overall ship/edit/reject decision.
• Normalizes judgments into a consistent output including a value, a 0–1 score, and judge confidence.
• Applies trust rules (based on confidence and/or distance from 0.5) and then:
  • Routes trusted rows to “Trusted Grades”
  • Sends “needs review” items to human-review

Use cases

• QA for LLM outputs: quickly verify responses against a reference answer before they reach users.
• Low-confidence triage: automatically flag uncertain grades for human inspection instead of blindly accepting AI outputs.
• SaaS operations & support: grade knowledge-base or support responses for relevance and clarity, then route only the safe ones forward.

Technical details

• Community node: n8n-nodes-judgment (Judgment API)
• Credential setup: create a Judgment API credential using your API key and select it in the Grade Answers step.
• Node logic used: if, set, code, no op, and sticky note to implement verdict mapping and routing.