Systems
01 / AI-assisted systems developer· Automation

A self-hosted support-triage pipeline: a local Ollama model routes tickets and drafts KB-grounded replies for human approval — every prompt change gated by an eval corpus.

Year
2026
Status
Public
Stack
n8n, Ollama, Eval-gated, Node
routing
88%
n8n canvas of a separate guardrails test workflow: test cases fan out to guardrail checks for jailbreaks, NSFW content, PII, secret keys and topic drift, each with pass and fail paths, collected by a results node
01

Support inboxes are repetitive but not quite scriptable — routing and drafting need judgment. Cloud AI could do it, but piping a support inbox through someone else's API means paying rent on your own mail, forever. A home rig can run the model.

02

A self-hosted n8n pipeline: inbound support email is triaged by a local Ollama model, which routes each ticket and drafts a reply grounded in a knowledge base. Nothing sends automatically — drafts land in an approval queue for a human. A live dashboard shows tickets, stats, and the pending-drafts count, with a daily digest email.

03

No prompt change ships on gut feel. A 40-case eval corpus gates the pipeline; AI-primary routing was promoted only when it beat the rules baseline — 88% against 70%. Ops is built for unattended running: exactly-once email processing, a keep-alive task at logon, nightly backups, and a stress-test suite that proves the recovery behaviour.

04

LOCAL OLLAMA > CLOUD APIa support inbox never leaves the machine
EVAL CORPUS GATE88% vs 70% baseline — measured, then shipped
HUMAN APPROVAL QUEUEAI drafts; a person sends