The fastest way to leak Customer A's data to Customer B is a shared RAG index.
It happens quietly. You build a retrieval chatbot, it works in the demo, and you ship it. Then a second customer signs up. You dump their documents into the same vector store. Now a query from Tenant A can pull back chunks written by Tenant B. The model summarizes both. Nobody notices until someone does.
This is the core problem with multi-tenant RAG. It is an access-control problem wearing an AI costume.
Why embeddings forget who owns the data
Vector search does not care about ownership. It ranks chunks by cosine similarity to the query. The index has no concept of tenant, role, or permission. A relevant chunk is a relevant chunk, even when it belongs to someone who should never see it.
Standard RAG retrieval looks like this. Embed the query. Find the top-k nearest chunks. Stuff them into the prompt. Generate an answer.
Notice what is missing. There is no authorization step between "find the chunks" and "show the user." The retriever is acting as an unauthenticated database query against everyone's documents at once.
The wrong fix: filter after retrieval
The first instinct is to retrieve broadly, then filter the results. Pull top-20, check each chunk's owner, drop the ones the user cannot see.
This is wrong for two reasons.
First, it leaks through ranking. If 18 of your top-20 belong to other tenants, you discard them and return 2 weak chunks. The answer quality collapses, and the collapse itself signals that other data exists.
Second, it leaks through cost and latency. You paid to embed, search, and rank documents the user was never allowed to touch. At scale you are scanning the whole corpus on every query.
Filtering after the fact treats authorization as a courtesy. It needs to be a constraint.
The right fix: enforce the boundary before search
Push the access-control filter into the query itself. The retriever should only ever see chunks the caller is allowed to see.
Most production vector stores support metadata filtering at query time. Tag every chunk with its `tenant_id` when you index it. Then make `tenant_id` a non-negotiable filter on every search, derived from the authenticated session, never from the user's input.
```
results = index.query(
vector=embed(question),
top_k=8,
filter={"tenant_id": session.tenant_id} # from auth, not the prompt
)
```
The filter comes from the verified session. The user cannot ask for another tenant's `tenant_id` because they never supply it. The boundary is enforced before similarity ranking, not after.
For stricter isolation, go further. Give each tenant its own namespace or index. Physical separation removes the chance that a missing filter exposes everyone. The tradeoff is more indexes to manage and pay for. Per-tenant namespaces are a reasonable default for most SMB workloads.
Don't forget the layers around retrieval
Retrieval is where the leak usually happens, but it is not the only door.
Row-level rules inside a tenant matter too. If your app has roles, a chunk visible to an admin may be off-limits to a contractor. Tag chunks with the same permission metadata your application already enforces, and filter on it the same way.
The model output is another door. Once a chunk is in the prompt, the model will use it. There is no asking it nicely to forget. Authorization has to happen before the context window, because the model is not a security boundary.
Logging is the quiet one. If you log full prompts for debugging, you are now storing tenant data in your observability stack. Scope those logs, or you have built a second copy of the problem.
What we actually do on these builds
We treat the retriever as a query against an access-controlled store, not a search engine over a pile of documents.
Tenant isolation is decided at indexing time, enforced at query time, and tested with a deliberate cross-tenant probe before launch. The probe is simple. Log in as Tenant A, ask a question only Tenant B's documents can answer, and confirm you get nothing. We run that test on purpose, every time.
This is the kind of work behind our RAG Chatbot builds. We have shipped 17 production products, 10 of them in our first 7 months as a 2-person team, and the founder is an ex-AWS Federal Solutions Architect, so isolation is not an afterthought. One support agent we built and run in production does its work without ever crossing a tenant boundary.
Pricing is fixed up front, you own the code, and we exit when it ships. See the pricing page for the RAG Chatbot scope.
If you are putting multiple customers behind one AI assistant, the access-control design is the part that matters most. Want a second set of eyes on yours? Book a free 30-minute call.