Anthropic Claude Sonnet Agent Reasoning Quality Scoring (n8n)

Score an agent’s reasoning trace with Claude Sonnet—then pass/fail it automatically

This n8n automation workflow scores the quality of an agent’s reasoning trace using Anthropic Claude Sonnet 4.6. It generates per-dimension quality scores with justifications and returns a passed / failed result based on configurable thresholds—so your automation can reliably decide whether an agent’s output is acceptable.

What this workflow does

• Receives inputs from a parent workflow call: reasoning_trace (array), the agent verdict, and raw_evidence.
• Applies quality thresholds by setting an overall reasoning quality threshold and a per-dimension threshold to define pass/fail criteria.
• Calls Anthropic Claude Sonnet 4.6 to score the trace and evidence across dimensions such as:
  • Evidence grounding
  • Circularity
  • Contradiction handling
  • Confidence calibration
• Extracts structured results (dimension scores, justifications, and the overall reasoning_quality value) from Claude’s JSON output.
• Evaluates thresholds and returns:
  • passed: true/false
  • flagged_dimensions listing any dimension scores at or below threshold
• Returns a merged result back to the calling workflow for downstream automation.

Use cases

• Quality gating for AI agents in n8n: reject weak or poorly grounded reasoning before acting.
• Audit-friendly evaluation: capture justifications per dimension for review and iteration.
• Policy enforcement: automatically fail outputs when contradictions or circular logic are detected.

Technical details

• Runs as a sub-workflow called by an Execute Workflow step in a parent workflow.
• Uses nodes including if, set, and merge, along with an execute workflow trigger structure.
• Uses the Anthropic Chat Model via an Anthropic credential for Claude Sonnet 4.6.
• Includes UI elements such as a sticky note for setup guidance and configuration.

Setup tip: Create/select an Anthropic credential, then configure overall_threshold and dimension_threshold in the configuration step to match your quality bar.