Semantic Router in n8n: pgvector, Embeddings & 93% Accuracy

01 Why use a router if you already have an LLM?

In the previous first article I showed why LLM-based routing is an anti-pattern. In short: putting rules such as “if it is about delivery → search the FAQ, if it is about price → search the catalog...” into an LLM prompt is unpredictable, has no useful metrics, and is difficult to A/B test.

In the Puramur sales agent, I needed to strictly separate five types of queries:

pricing — “how much is it?”, “price of PR243587” — go to an SQL tool with live prices
faq — “when will you deliver?”, “how can I pay?” — use RAG over a view containing FAQ chunks
handoff — “I want to talk to a manager”, “give me an operator” — go directly to the handoff adapter
off_topic — “tell me a joke”, “what is the weather?” — get a soft redirect
sales — everything else — goes to an eight-stage sales state machine

Each route uses different tools, different prompts, and different data sources. Route classification is a critical point in the funnel. A mistake here means the customer gets the wrong response instead of the right one.

Main idea

A semantic router is a fast, deterministic classifier, which runs before the LLM. It moves the “which function should be called?” decision out of the probabilistic world of the LLM and into structured SQL, where it can be measured, tested, and improved with numbers.

02 Concept: embedding + cosine + threshold

Semantic routing is a classification problem. Given a user query, we need to return one of N routes. The classic ML solution is to train a classifier on labeled examples. But for five routes and a few hundred examples, that is overkill. A simpler approach is:

For each route, store several seed phrases (utterances) — typical queries for that route
Convert each phrase into a vector (embedding) through the OpenAI API
When a new query arrives, convert it into a vector as well
Find the nearest seed phrase using cosine similarity in pgvector
If similarity is above the threshold, the route is found. Otherwise, use the fallback route (sales in our case)

This is a k-nearest-neighbor classifier with k=1. It is simple, fast, requires no training, and makes it easy to add new phrases.

		User: "what is the price of Exotic Spa?"

		  ↓

		OpenAI Embed → [0.023, -0.145, 0.087, ...]  (1536 dims)

		  ↓

		Postgres pgvector cosine search:

		    "how much is it?" (pricing)     → 0.782

		    "shampoo price" (pricing)       → 0.756

		    "when will you deliver?" (faq)  → 0.412

		    "I want a manager" (handoff)    → 0.223

		  ↓

		Threshold check: 0.782 >= 0.55 (pricing threshold)

		  ↓

		Route: pricing (confidence: 0.782)

03 Architecture

In Puramur, the router is a dedicated Postgres table plus one n8n workflow. Everything runs on the existing infrastructure:

Postgres 15 + pgvector (the vector-search extension) — on Railway
n8n on Railway — orchestration
OpenAI text-embedding-3-small — 1,536-dimensional vectors, $0.02 per 1M tokens

There are no separate services such as Pinecone, Weaviate, or Qdrant, and no separate ML models. Everything lives in the same Postgres database next to the sales agent. This matters because the router and the agent read from the same database without extra network hops.

04 puramur_router_utterances table schema

CREATE EXTENSION IF NOT EXISTS vector;

		CREATE TABLE puramur_router_utterances (
		  id           BIGSERIAL PRIMARY KEY,
		  route_name   TEXT NOT NULL,
		  utterance    TEXT NOT NULL,
		  embedding    vector(1536) NOT NULL,
		  priority     INT NOT NULL DEFAULT 10,
		  threshold    REAL NOT NULL DEFAULT 0.55,
		  created_at   TIMESTAMPTZ NOT NULL DEFAULT NOW(),
		  updated_at   TIMESTAMPTZ NOT NULL DEFAULT NOW()
		);

		-- Index for fast cosine search
		CREATE INDEX idx_utterances_embedding 
		  ON puramur_router_utterances 
		  USING hnsw (embedding vector_cosine_ops);

		-- Index for filtering by route
		CREATE INDEX idx_utterances_route ON puramur_router_utterances(route_name);
		

Three details are worth explaining.

vector(1536) — the size of the OpenAI embedding text-embedding-3-small. If you change the model, you need to change the vector size and re-embed all utterances because models with different dimensions are not compatible.

HNSW index — Hierarchical Navigable Small World, an ANN structure for fast cosine search in high-dimensional spaces. With 50 utterances in the table, the difference from a full scan is negligible. With 5,000, HNSW can be roughly 100× faster.

threshold per row — may look redundant (why store a threshold on every phrase if it is usually the same within a route?). The reason is tuning: you may want to experiment with individual phrases. One utterance may be a hard case that needs a higher threshold to avoid false positives. Keeping the threshold in the table gives you flexibility without migrations.

05 Seed workflow: 48 phrases with batch embedding

The next step is to populate the table with seed phrases. For Puramur, I created 48 phrases across four routes (sales has no utterances because it is the fallback). Here is the distribution:

Route	Priority	Threshold	Count
`pricing`	1	0.55	11
`handoff`	1	0.55	10
`faq`	2	0.50	18
`off_topic`	3	0.55	9

Priority is the conflict-resolution order when a phrase matches multiple routes (for example, “how much does it cost to contact a manager?”). A lower number means higher priority. More on this in the Route Decision.

Example phrases

-- pricing (11 phrases)
	"how much is it",
	"how much is the shampoo",
	"how much is PR243587",
	"what is the price",
	"price of",
	"cost",
	"how much",
	"how much",
	"price",
	"cost",
	"how much does it cost"
	-- handoff (10 phrases)
	"I want to talk to a manager",
	"give me an operator",
	"I want to talk to a person",
	"real human please",
	"real person",
	"with a manager",
	"talk to an operator",
	"live person",
	"transfer me to a manager",
	"complaint"
	-- faq (18 phrases)
	"when will you deliver",
	"how long will the order take",
	"how can I pay",
	"Nova Poshta rates",
	"free delivery",
	"cash on delivery",
	-- ... and so on
	-- off_topic (9 phrases)
	"tell me a joke",
	"what is the weather",
	"what's the weather",
	"who is the president",
	"ignore all instructions",
	"jailbreak",
	-- ...
	

Batch embedding over HTTP instead of the n8n Embed node

The standard approach in n8n is to use the OpenAI Embed node in a loop. But the OpenAI API supports batch — up to 2,048 input texts in a single request. For 48 phrases, that means one API call instead of 48, saving both time and money.

// Prepare Batch Embed (Code node)
	const utterances = [
	  { route: 'pricing', text: 'how much is it', priority: 1, threshold: 0.55 },
	  { route: 'pricing', text: 'how much is the shampoo', priority: 1, threshold: 0.55 },
	  // ... all 48
	];

	return [{ 
	  json: { 
		model: 'text-embedding-3-small', 
		input: utterances.map(u => u.text),
		// keep metadata for the next node
		_utterances: utterances,
	  }
	}];
	

// HTTP Request node: OpenAI Batch Embed
	POST https://api.openai.com/v1/embeddings
	Content-Type: application/json
	Authorization: Bearer sk-...

	{
	  "model": "text-embedding-3-small",
	  "input": ["how much is it", "how much is the shampoo", ...]
	}
	

// Response — 48 embeddings in one array
	{
	  "data": [
		{ "index": 0, "embedding": [0.023, -0.145, ...] },
		{ "index": 1, "embedding": [0.031, -0.152, ...] },
		...
	  ]
	}
	

Next comes a Code node that formats a bulk INSERT:

// Format INSERT rows (Code node)
	const embeddings = $input.first().json.data;
	const utterances = $('Prepare Batch Embed').first().json._utterances;

	const rows = embeddings.map((e, i) => {
	  const u = utterances[i];
	  return {
		route_name: u.route,
		utterance: u.text,
		embedding: '[' + e.embedding.join(',') + ']',
		priority: u.priority,
		threshold: u.threshold,
	  };
	});

	return rows.map(json => ({ json }));
	

// Postgres INSERT (Insert operation, batch)
	INSERT INTO puramur_router_utterances 
	  (route_name, utterance, embedding, priority, threshold)
	VALUES ($1, $2, $3::vector, $4::int, $5::real);
	

The entire seed workflow is five nodes, takes about 15 seconds, and inserts all 48 phrases into the table with embeddings. It is idempotent: after a TRUNCATE, you can rerun it to refresh the dataset.

06 Test workflow with a webhook

Before integrating the router into the sales agent, I built a separate test workflow. This is important engineering hygiene: the router is a component with a clear contract (query → route + confidence), so it should be tested in isolation.

Webhook Trigger (POST /webhook/router-test)
		↓
	Prepare Embed (Code) ← take query or queries[] from body
		↓
	OpenAI Batch Embed (HTTP) ← 1 API call for all queries
		↓
	Split Queries (Code) ← split into items
		↓
	Similarity Search (Postgres) ← cosine search LIMIT 10 per query
		↓
	Route Decision (Code) ← priority + threshold logic
		↓
	Aggregate Results (Code) ← summary metrics
		↓
	Respond to Webhook
	

Prepare Embed

// Accept either {query: "..."} or {queries: [...]}
	const body = $input.first().json.body || $input.first().json;

	let queries;
	if (body.query) queries = [body.query];
	else if (Array.isArray(body.queries)) queries = body.queries;
	else throw new Error('Expected {query} or {queries[]}');

	return [{
	  json: {
		model: 'text-embedding-3-small',
		input: queries,
		_queries: queries,
	  }
	}];
	

Similarity Search

-- Postgres executeQuery mode, runOnceForEachItem
	SELECT
	  route_name,
	  priority,
	  threshold,
	  utterance,
	  1 - (embedding <=> $1::vector) AS similarity
	FROM puramur_router_utterances
	ORDER BY embedding <=> $1::vector ASC
	LIMIT 10;
	

The operator <=> — is cosine distance in pgvector. The calculation is simple: 1 - distance = similarity. Values are in the [-1, 1] range, but for normal text they are almost always in [0, 1]; negative values are rare with modern embedding models.

LIMIT 10 — we take the top 10 because the top 1–3 results may all belong to the same route, while Route Decision may need the best result per route to analyze alternatives. Ten is a convenient number for debugging.

07 Route Decision: priority-based multi-match

This is the most important Code node in the router. It receives 10 rows from the similarity search and must return one route.

// Route Decision (Code node, runOnceForAllItems)
	const allRows = $input.all().map(item => item.json);

	if (allRows.length === 0) {
	  return [{ json: { route: 'sales', confidence: 0, reason: 'no_utterances' } }];
	}

	// Step 1: take the best score per route
	const bestPerRoute = {};
	for (const row of allRows) {
	  const sim = parseFloat(row.similarity);
	  if (!bestPerRoute[row.route_name] || bestPerRoute[row.route_name].similarity < sim) {
		bestPerRoute[row.route_name] = {
		  route: row.route_name,
		  priority: parseInt(row.priority),
		  threshold: parseFloat(row.threshold),
		  similarity: sim,
		  utterance: row.utterance,
		};
	  }
	}

	// Step 2: keep routes that passed the threshold
	const matched = Object.values(bestPerRoute)
	  .filter(r => r.similarity >= r.threshold);

	// Step 3: sort by priority (asc), then similarity (desc)
	matched.sort((a, b) => {
	  if (a.priority !== b.priority) return a.priority - b.priority;
	  return b.similarity - a.similarity;
	});

	// Step 4: winner or default sales
	const decision = matched.length > 0
	  ? { 
		  route: matched[0].route, 
		  confidence: Math.round(matched[0].similarity * 1000) / 1000,
		  matched_utterance: matched[0].utterance,
		  reason: 'threshold_met_priority_' + matched[0].priority,
		}
	  : { 
		  route: 'sales', 
		  confidence: 0, 
		  matched_utterance: null,
		  reason: 'no_threshold_met_default_sales',
		};

	return [{ json: { ...decision, alternatives: matched } }];
	

The logic breaks down into three sequential steps. Best-per-route prevents a situation where pricing has five phrases in the top 10 even though only the strongest one matters. The threshold filter removes weak matches. The priority sort resolves conflicts when a query matches multiple routes above their thresholds.

Conflict example

Query: “how much does it cost to contact a manager?”
Matches:
• pricing: 0.61 (>= 0.55 ✓, priority 1)
• handoff: 0.58 (>= 0.55 ✓, priority 1)

Both pass the threshold. Their priority is the same. The winner is the one with the higher similarity — pricing. That may be wrong, but in practice these edge cases are rare and can be handled by adding a more precise seed phrase.

08 The day the router delivered 6.7%

I assembled a test batch of 30 queries across different routes, labeled with the expected route, and ran them through the webhook. The result:

6.7%

Match rate

2/30

Passed threshold

Fell to default

Only 2 of 30 queries passed the threshold. All the others fell back to sales. That was catastrophic — worse than random, because even random assignment would give about 25% accuracy across the four non-sales routes.

My initial thresholds were:

pricing:   0.70
	faq:       0.65
	handoff:   0.75
	off_topic: 0.75
	

It looked “reasonable”: if embedding cosine values are in the [0, 1] range, then 0.7+ seems like a sensible threshold for a confident match. It turned out to be an optimistic assumption with no data behind it.

My first reaction was to frantically look for a bug. Maybe the Route Decision code was wrong? Maybe the embeddings had a different dimensionality? Maybe pgvector cosine distance was not returning what I thought? I spent 30 minutes debugging and manually comparing cosine similarity in Python (numpy.dot(a, b) / (norm(a) * norm(b))) with what pgvector returned. The values matched to four decimal places.

So the bug was not in the code. The bug was in my assumptions about the expected similarity values.

09 Confusion matrix — metrics instead of guesswork

Instead of guessing, I decided to measure. I wrote diagnostic SQL that shows the actual distribution of similarity values in my dataset:

-- Within-route: how close are phrases from the SAME route
	SELECT
	  a.route_name,
	  MIN(1 - (a.embedding <=> b.embedding)) AS min_sim,
	  AVG(1 - (a.embedding <=> b.embedding))::numeric(4,3) AS avg_sim,
	  MAX(1 - (a.embedding <=> b.embedding)) AS max_sim
	FROM puramur_router_utterances a
	JOIN puramur_router_utterances b 
	  ON a.route_name = b.route_name AND a.id != b.id
	GROUP BY a.route_name;
	

Route	MIN	AVG	MAX
pricing	0.412	0.634	0.812
faq	0.298	0.521	0.784
handoff	0.445	0.687	0.831
off_topic	0.312	0.598	0.795

Now this is interesting. Within the same route, average similarity is 0.5-0.7, while the maximum is 0.79–0.83. In other words, “how much is it” and “how much is the shampoo” are not at 0.95 as I had assumed, but around 0.71.

Next comes cross-route similarity: how similar phrases from different routes are to one another (this is what causes false positives).

-- Cross-route: how close are phrases from DIFFERENT routes
	SELECT
	  a.route_name AS route_a,
	  b.route_name AS route_b,
	  MAX(1 - (a.embedding <=> b.embedding))::numeric(4,3) AS max_sim,
	  AVG(1 - (a.embedding <=> b.embedding))::numeric(4,3) AS avg_sim
	FROM puramur_router_utterances a
	JOIN puramur_router_utterances b 
	  ON a.route_name < b.route_name
	GROUP BY a.route_name, b.route_name
	ORDER BY max_sim DESC;
	

Route A	Route B	MAX	AVG
pricing	faq	0.539	0.312
faq	handoff	0.478	0.267
pricing	handoff	0.412	0.234
handoff	off_topic	0.398	0.241
faq	off_topic	0.371	0.198
pricing	off_topic	0.298	0.156

The key number here is max cross-route similarity 0.539. This is the similarity between the closest phrases from different routes. That means my “safe zone” for thresholds is [0.55, 0.70]:

Lower bound 0.55 — slightly above the cross-route maximum of 0.539, to avoid false positives
Upper bound 0.70 — slightly below the within-route maximum of about 0.80, so real matches can still pass

My thresholds of 0.70/0.65/0.75/0.75 were above the average within-route range. In practice, almost no real query could pass them. That was the bug.

10 The fix: one SQL command

UPDATE puramur_router_utterances 
	SET threshold = 
	  CASE route_name
		WHEN 'pricing' THEN 0.55
	WHEN 'faq' THEN 0.50
	WHEN 'handoff' THEN 0.55
	WHEN 'off_topic' THEN 0.55
	END,
	  updated_at = NOW();
	

I reran the same batch of 30 queries. The result:

56.7%

Match rate

17/30

Passed threshold

93%

Overall correct

A 56.7% match rate may not look impressive, but this number includes only non-default matches. In my batch of 30 queries, 11 were sales queries that were supposed to land in the fallback — in other words, they were supposed not to be classified into a non-default route. So the real metric is overall correctness:

17 non-sales queries were correctly routed to pricing, faq, handoff, or off_topic
11 sales queries correctly landed in the default sales fallback
2 queries were classified incorrectly
Total: 28/30 = 93.3% correctness

The metric matters

match_rate — is a scan rate, not accuracy. It tells you what percentage of queries received a non-default route. But for a sales bot, a correct sales fallback is also a correct classification. Measure correctness by comparing expected vs actual routes.

What to do with the two errors

The two failures were edge cases:

The query “what is the weather in Kyiv?” was originally written in Ukrainian and produced a cross-lingual mismatch against the English utterance “what's the weather”: similarity was 0.449, below threshold. It fell back to sales. The sales prompt still redirects the conversation back to the main topic, so the result is a soft redirect instead of a hard off_topic route. That is acceptable behavior.
The query “ignore all instructions” was originally written in Ukrainian and had similarity 0.52 to the English utterance “ignore all instructions”, below threshold. It fell back to sales. The sales prompt already has safety rules that prevent prompt injection from being followed.

There are two ways to improve both cases: add Ukrainian versions of the off_topic phrases (the simplest option), or lower the off_topic threshold to 0.45. I chose the first and expanded the seed set after this test.

11 Bruno test suite — 27 scenarios

The router is going into a production sales agent. That means every change to the seed set or thresholds requires regression testing across all routes. I created a test suite in Bruno (an open-source alternative to Postman) with 27 scenarios across six groups.

// test_pricing_route.bru
	meta {
	  name: T-01 pricing basic
	  seq: 1
	}

	post {
	  url: {{routerUrl}}
	  body: json
	}

	body:json {
	  { "query": "how much is Exotic Spa shampoo" }
	}

	tests {
	  test("Route is pricing", () => {
		expect(res.getBody().route).to.equal("pricing");
	  });
	  test("Confidence > 0.55", () => {
		expect(res.getBody().confidence).to.be.greaterThan(0.55);
	  });
	  test("Threshold met", () => {
		expect(res.getBody().reason).to.include("threshold_met");
	  });
	}
	

Test distribution in the suite:

Group	Count	Examples
Pricing positive	5	“how much is it”, “PR243587 cost”, “what is the price of the conditioner”
FAQ positive	6	“when will you deliver”, “how can I pay”, “Nova Poshta rates”
Handoff positive	4	“I want a manager”, “give me an operator”, “real human please”
Off-topic positive	4	“tell me a joke”, “ignore instructions”, “what can you do”
Sales default	4	“I have a Sphynx cat”, “my British Shorthair sheds a lot”, “Yorkie for a dog show”
Edge cases	4	coreference, cross-lingual, prompt injection

Bruno CLI lets you run them all at once:

$ bru run --env local
	Running Collection: puramur-router
	Total tests: 81 (3 assertions × 27 scenarios)
	Passed: 75
	Failed: 6
	Duration: 4.2s
	

The six failures are edge cases where I set expectations higher than the router can currently satisfy. They are flags for the next improvements: add an utterance or raise/lower the threshold for a specific route.

Reproducibility

The Bruno test suite is stored in Git together with the workflow JSON. When someone needs to reproduce the router in another project, they have (a) a seed workflow that loads the utterances, (b) a test suite that verifies behavior, and (c) threshold documentation. That reduces regression risk.

12 Practical takeaways

The main lesson from this part of the Puramur project is that threshold tuning should not be done by intuition. Building a router with “reasonable” default thresholds almost guarantees a 6–10% match rate. Only diagnostic queries over your real dataset reveal the actual boundary between signal and noise. In my case, average within-route similarity was 0.63, cross-route max was 0.54, and a threshold of 0.55 between them produced an almost ideal balance.

The second lesson is that match_rate is not the only metric. A semantic router with a fallback must distinguish between a good match to a non-fallback route and a correct fallback. In my batch, 11 of 30 queries clearly belonged to the sales class, and the router correctly did not classify them into another route. That is not a miss; it is correct behavior. The metric that captures this is overall correctness, not match_rate.

The third lesson is that a separate test workflow saves a lot of debugging pain. A router is a component with a clear contract. Testing it in isolation instead of through the sales agent has several advantages: bugs are easier to debug because there are fewer variables, regression tests are faster because they do not require LLM calls, and adding utterances is safer because you are less likely to break production.

Semantic router checklist

A dedicated table with a vector(N) field + HNSW index
Threshold per row (flexibility without migrations)
Priority per route (for match conflicts)
Batch embedding when seeding (1 API call for 48 phrases, not 48 calls)
A separate test workflow with a webhook
Diagnostic SQL queries for within-route / cross-route similarity
Threshold tuning based on a confusion matrix, not intuition
Metric: overall correctness (correct match + correct fallback), not just match_rate
Bruno test suite with grouped scenarios
Route Decision code with priority-based multi-match resolution

Next article in the series

#4 — 5 n8n bug patterns that break AI workflows
A concise practical checklist with real examples: AI Agent input passthrough, Postgres output replacing the input, Merge combineByPosition hangs, the Always Output Data toggle, and IF-branch merging pitfalls.

Read the next article →

n8n Semantic Router: From 6.7% to 93% Accuracy