Multi-tenant RAG isolation: pushing the tenant boundary into the storage engines

Why WHERE tenant_id = ? is filtering, not isolation

The first question any CISO asks about a multi-tenant retrieval platform is how you stop one tenant's documents reaching another tenant's answers. The common response points at the application layer: middleware parses a token, extracts a tenant id, and the ORM appends a predicate to every query.

That is filtering. It works exactly as long as every query, in every code path, written by every engineer who will ever touch the codebase, remembers to carry the predicate. The failure modes are ordinary rather than exotic:

• A library upgrade turns a malformed token from a hard error into a warning, and the request proceeds with a null tenant id.
• A raw SQL query written for a reporting endpoint bypasses the ORM layer that carried the filter.
• A complex join causes the ORM to drop the predicate on a subquery.
• A background job — reindexing, migration, an export — runs without a request context at all.

None of these is a bad engineer. They are the statistical outcome of a security model that has to be re-implemented correctly on every future line of code. In a RAG system the consequence is worse than a normal data leak: leaked rows do not just appear in a response, they get assembled into a prompt, and the model summarises a competitor's data into the reader's own workflow. There is no obviously wrong-looking output to catch it.

The alternative is to move the boundary underneath the application, into the engines that hold the data, so that a flaw in the application produces a visible break instead of a silent leak. In EinCoreRAG we did that at two layers independently — relational and vector — so that a failure at one cannot reach the other's data. The companion podcast episode covers the reasoning; this post is the implementation detail, including the parts that bite.

Layer 1 — PostgreSQL Row-Level Security

The mechanism is small. The traps around it are not.

ALTER TABLE documents ENABLE ROW LEVEL SECURITY;
ALTER TABLE documents FORCE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON documents
  USING (tenant_id = current_setting('app.current_tenant_id', true)::uuid);

Every transaction opens by setting the tenant for that transaction only:

SET LOCAL app.current_tenant_id = '…';

Four details in those five lines carry the whole design.

SET LOCAL, not SET. Any production deployment puts a connection pooler in front of Postgres, and a pooler multiplexes many client requests over a few long-lived backend connections. A plain SET writes to the session, which outlives the request and belongs to whoever gets that connection next. SET LOCAL scopes the value to the transaction and Postgres clears it on commit or rollback, which leaves the connection clean for the next borrower. If you run PgBouncer, this also means you need transaction-level pooling and all your work inside an explicit transaction — in statement pooling mode SET LOCAL and the statements that depend on it can land on different backends.

The true in current_setting. That is the missing_ok flag. Without it, a request that forgot to set the variable raises an error; with it, the lookup returns NULL. Comparing tenant_id to NULL is never true, so the policy matches no rows. The query succeeds and returns nothing. That is the fail-closed property, and it is deliberate: the loud, empty result is the safe outcome, and an empty result where data was expected is something monitoring and tests can both see.

FORCE ROW LEVEL SECURITY. ENABLE alone does not apply policies to the table's owner. Applications very often connect as the role that owns the schema, because that is what the migration tool was configured with — and that role reads every row as if RLS were not there. This is the single most common way a team ships RLS, tests it as a superuser, sees it "work", and has no isolation at all in production. Either connect as a non-owner role with only DML rights, or add FORCE, or both. Roles holding the BYPASSRLS attribute are exempt from policies too, and superusers always are — so no application should ever connect as one.

SECURITY DEFINER functions are a hole you dig yourself. A helper function — a billing aggregate, a search wrapper — created with SECURITY DEFINER runs with the privileges of the role that created it, typically an administrative role. Calling it from application code executes the body outside the caller's RLS context. Default to SECURITY INVOKER; where a definer function is genuinely required, set an explicit search_path on it and treat it as a reviewed security boundary, not as a utility.

Make the guarantee testable

An isolation property that no test can falsify is a belief. The test that matters is deliberately adversarial: open a transaction as tenant A, run the exact queries the application runs, and assert that tenant B's rows are absent — then repeat with the session variable deliberately unset and assert the result is empty rather than complete.

BEGIN;
SET LOCAL app.current_tenant_id = '<tenant-a>';
SELECT count(*) FROM documents;          -- expect: only A's rows
ROLLBACK;

BEGIN;                                    -- no SET LOCAL at all
SELECT count(*) FROM documents;          -- expect: 0, not everything
ROLLBACK;

Run it in CI against a database seeded with at least two tenants, and run it as the application's role, not as the migration role. A test that passes as a superuser proves nothing.

Layer 2 — the vector store

Qdrant, like most vector databases, offers two ways to separate tenants: one shared collection with a tenant_id payload filter, or a collection per tenant. We started on the shared collection and moved off it. The reasoning, and the cost of the move, both belong in the open:

Why we moved. A payload filter is the same architecture as WHERE tenant_id = ? — the boundary is enforced by the query author. A wrong or missing filter returns the wrong tenant's neighbours. With a collection per tenant, a query aimed at the wrong place gets a 404 from the engine rather than someone else's documents. Smaller HNSW graphs also reach their neighbours in fewer hops, and a collection can be pinned to a specific node or region when one regulated tenant has residency requirements the rest do not.

What it costs. This is not free, and a team copying the pattern should budget for it. Each collection carries its own index structures and its own memory floor, so a long tail of small tenants is markedly less efficient than one shared graph — the shared-collection design exists for good reasons at high tenant counts. Provisioning, backup, migration and schema changes now iterate over collections instead of running once. Multi-tenant analytics across the whole corpus become a fan-out. If your tenant count is in the thousands and your compliance surface is low, the shared collection with a payload filter and a hard-reviewed query layer may well be the better trade — but then be precise with your customers that what you have is filtering.

What "fail-closed" means here, concretely

• The API forgets SET LOCAL → the RLS policy compares against NULL → zero rows.
• A query reaches the wrong collection name → the engine returns 404, not another tenant's vectors.
• Both layers fail at once → retrieval yields nothing, so there is no cross-tenant context in the prompt for the model to synthesise.

Each failure is visible in logs and in the answer quality, rather than silent.

What this does not protect against

Two engine-level boundaries close one class of failure. They are not a data-protection story on their own, and presenting them as one is where this kind of architecture gets oversold:

• Anything outside the query path. Backups, restores, replicas, exports, analytics pipelines and admin tooling frequently run as roles that bypass RLS. Every one of them needs its own answer.
• Logs and traces. Prompt and retrieval logging is how you debug a RAG system and how you meet incident-reconstruction duties — and it is a second copy of tenant content, usually in a store with none of these controls.
• Caches and shared derived state. An embedding cache, a reranker cache or a semantic cache keyed on content rather than on content-plus-tenant will happily serve one tenant a result computed for another.
• Assembly in the application. If the orchestration layer retrieves correctly per tenant and then merges contexts — for a comparison feature, a "related documents" panel, an agent that queries twice — isolation ends where the engine's responsibility ends.
• Training and fine-tuning. Data separated at rest and mixed in a training set is no longer separated.
• Injection within a tenant. These controls say nothing about a malicious document inside one tenant's own corpus steering that tenant's agent.
• The people and the platform. Operators with database access, and the hosting model itself, sit outside this boundary entirely. That is a different conversation — the one about where the data actually runs.

A short checklist

1. The application connects as a role that owns nothing and holds no BYPASSRLS.
2. ENABLE and FORCE ROW LEVEL SECURITY on every tenant-scoped table.
3. SET LOCAL inside an explicit transaction; pooler in transaction mode.
4. current_setting(…, true) so a missing tenant fails closed rather than erroring.
5. SECURITY INVOKER by default; every definer function reviewed and search_path-pinned.
6. A CI test that asserts empty with no tenant set, run as the application role.
7. Vector isolation chosen deliberately — engine-enforced or filter-enforced — and described to customers in the words that are actually true.
8. A written list of every path that reads tenant data outside the request cycle, each with its own control.

Retrofitting any of this into a busy application means walking every query, every migration and every job. It is dramatically cheaper on day one than in month eighteen — which is the real reason to decide it early, before there is a tenant whose data you would be moving.

This post has been redacted by our automated security pipeline to remove internal topology details.