n8n AI Fact Checker: Ollama + Wikipedia Score PASS/FLAG

Get factual PASS/FLAG results for every AI claim—verified against your facts and Wikipedia

This n8n AI Fact Checker workflow takes any AI output you send to it and returns a strict PASS, FLAG, or BLOCK verdict for each factual claim—backed by quotes from your trusted facts and Wikipedia. It runs on local Ollama (no API key), fails closed when the model is down, and emails a weekly truth score.

What this workflow does

• Receives AI output via a POST webhook (or uses a built-in sample) and reads all settings from Read Request & Config.
• Extracts checkable claims using a local Ollama model: it targets names, numbers, dates, and events while ignoring opinions.
• Collects evidence for each claim by first matching your trusted facts from the Data Table truth_facts, then pulling matching extracts from Wikipedia. The search subject comes from the claim text (not the model).
• Judges each claim with citations: a judge model rates each claim VERIFIED, UNSUPPORTED, or CONTRADICTED and must include a quote for the verdict.
• Enforces strict validation in code: every quote must exist word-for-word in the evidence (including numbers). If the model fails, results become FLAG with a truth score of 0.
• Produces PASS/FLAG/BLOCK: contradictions cause BLOCK, too many unsupported claims cause FLAG, otherwise PASS.
• Logs and notifies: returns verdicts, records checks in truth_checks, and emails alerts on FLAG/BLOCK. Every Monday at 08:00, it emails a weekly truth score per AI source and deletes checks older than 90 days.

Use cases

• Validate factual sections in AI-generated customer emails or SaaS support answers.
• Risk-control for content workflows: automatically flag unverifiable claims before publishing.
• Monitor reliability over time with a weekly truth score by AI source.

Technical details (n8n)

• Webhook (POST) for incoming AI output
• Ollama (local) for claim extraction and judging (no external API key)
• Data tables: truth_facts and truth_checks
• Nodes used: if, code, webhook, data table, email send, sticky note