Most RAG stacks pay for a database they barely use.
OpenSearch Serverless is the default vector store for a lot of teams. AWS pushes it, the tutorials use it, and it works on day one. Then the bill arrives. OpenSearch Serverless charges by OCU, the compute unit it keeps running whether you query it or not. There is a minimum floor. For a chatbot that gets a few hundred questions a day, you pay for capacity that sits idle most of the hour.
S3 Vectors changes the math. It stores vectors in S3 and charges for storage plus the queries you actually run. No always-on compute floor. For spiky or low-volume workloads, that is the whole ballgame.
When the migration pays off
The win shows up under three conditions. Hit all three and you should move.
Your query volume is low or bursty. If traffic comes in waves, an internal support tool, a docs assistant, a tool used during business hours, you are paying OpenSearch to idle overnight. S3 Vectors bills per query. Idle costs almost nothing.
Your corpus is large but cold. Millions of chunks that rarely change. S3 storage is cheap. Keeping all of that warm in OpenSearch memory is not.
Your latency budget has slack. S3 Vectors is fast enough for most retrieval, but it is object storage underneath. Expect tens to low hundreds of milliseconds, not the single-digit response you get from a warm in-memory index. For a chatbot that already waits on an LLM generation, that added retrieval time disappears into the noise.
That last point matters. The support agent we built and run in production has its human-felt latency dominated by the human-in-the-loop step, so shaving 80 milliseconds off retrieval is not the constraint. Spend the engineering effort on the bill instead.
When it does not
Do not move if you are doing high-QPS retrieval with a tight latency SLA. A user-facing search box that fires on every keystroke needs a warm index. S3 Vectors will feel slow there, and the per-query cost can flip against you at high volume.
Do not move if you depend on OpenSearch features beyond vector search. Hybrid keyword-plus-vector ranking, complex filters, faceting, aggregations. S3 Vectors is a vector store, not a search engine. If you are using OpenSearch as a real search engine, it is earning its keep.
And do not move for the sake of moving. The migration is real work. You re-embed or re-export, you rewrite the retrieval client, you re-test recall. If the OpenSearch bill is a rounding error on your AWS spend, leave it alone.
What the migration actually involves
The work is bounded and predictable. Export your vectors and metadata. Create S3 Vector indexes with the same dimensions and distance metric. Bulk-load. Swap the retrieval call in your app from the OpenSearch client to the S3 Vectors API. Re-run your eval set and confirm recall holds.
The part teams underestimate is the eval set. You need a fixed list of test questions with known-good answers before you touch anything. Without it, you cannot prove the new store retrieves as well as the old one. Most teams skip this and then argue about whether quality dropped. Build the eval first.
The other trap is distance metric mismatch. If your embeddings were tuned for cosine and you load with the wrong metric, recall quietly degrades. It still returns results. They are just worse. The eval set catches this. Nothing else will.
The honest cost picture
Most AWS bills hide savings nobody is tracking, and idle vector compute is a common place it sits. We see OpenSearch Serverless running 24/7 to serve a workload that peaks for two hours. Moving that to S3 Vectors is one of the cleaner cuts available, because the retrieval quality stays the same and only the billing model changes.
We have shipped 17 production products, 10 of them in our first 7 months as a 2-person team, several of them RAG systems on AWS. The pattern repeats. The first version ships on OpenSearch because it is the path of least resistance. The bill forces a second look. Sometimes the move is obvious. Sometimes OpenSearch is the right tool and the answer is no. The point is to decide on the numbers, not the default.
If you want a fast read on your own stack, our Cloud Bill Audit finds where the spend hides. If the RAG itself needs building or fixing, our RAG Chatbot work is fixed price, you own the code, and we exit. No retainer, no lock-in.
Want a second opinion before you commit to the migration? Book a free 30-minute call and we will tell you straight whether the move pays off for your workload.