Client and server versions do not match
The Python SDK was upgraded in CI without pinning, the server image is still on an older tag, and every call either fails outright or quietly ignores a new parameter.
Client and server pinned together
Building RAG on Chroma?
A prototype that ran out of PersistentClient on a laptop has become a production retrieval layer, the server-mode container keeps reporting a client API version it does not speak, and a collection that held forty thousand vectors last week now returns zero matches without any error in the log. We walk the storage layout, check the SDK and server versions against each other, and turn the retrieval back into something a chatbot can rely on.
What it is
Chroma is an open-source embedding database written in Python with a Rust core for the index. It is designed to be the smallest sensible thing between an application and a vector store, so RAG prototypes can start with an in-memory client and move to a persistent store or a server deployment without changing the application code. Collections hold embeddings, documents, IDs and metadata together, and queries combine vector similarity with structured filters.
The project was created by Jeff Huber and Anton Troynikov in 2022 and has since grown a managed cloud offering alongside the open-source code. Chroma's defaults are forgiving, which is part of the appeal and part of the trouble, because teams often ship with the ephemeral in-memory client, with default embedding functions that do not match the ingest pipeline, or with persist_directory pointing at a container path that is wiped on redeploy. Client and server versions moved rapidly between 0.4, 0.5 and 0.6, so a mismatched SDK is one of the single most common failure modes in the wild.
The honest version
No technology is all upside. This is where Chroma earns its place and where it tends to cause trouble.
Sound familiar?
The Chroma problems that bring people to us, and what you have once each one is fixed.
The Python SDK was upgraded in CI without pinning, the server image is still on an older tag, and every call either fails outright or quietly ignores a new parameter.
Client and server pinned together
A Chroma server started inside a container with the default ephemeral storage. Every restart wipes the collection and the ingest job has to run from scratch while users see empty answers.
Persistence on durable storage
Multiple ingest workers hammering upsert in parallel, long-running queries holding read locks, or a backup script reading the file at the wrong moment. SQLite does not fan out.
Writes serialized or moved off SQLite
The collection was built with an onnx MiniLM default, the query path has been upgraded to text-embedding-3-small, and the two coordinate spaces have nothing to do with each other.
One embedding function across read and write
A collection whose index file was lost during a volume move, or a restore that pulled the SQLite file without the parquet shards next to it. Queries error until the index is rebuilt.
Index rebuilt, storage layout understood
A server upgrade introduced tenants and databases, and the client is still calling the old flat API. Nothing is actually broken, the namespacing just moved.
Client talks to the correct tenant
A collection created with construction parameters copied from a different engine, where the numbers are valid but mean something else, so the index will not build.
HNSW parameters measured on your corpus
A batch that is too big for the server's memory, or a payload with document text that pushes past the request limit. The server returns 500 and the worker retries the same batch.
Batching sized to the server
What we do
Pick up the phone for one broken collection, or let us run retrieval end to end.
How it works
Tell us what's going wrong. We look at it, then tell you plainly what's wrong and what it will cost.
We fix the cause, not the symptom. If something is down, getting it back up comes first.
Backups, monitoring, updates and access, so the same problem can't quietly come back.
You get a plain-language write-up. Or we stay on for hosting, upkeep and new work.
Often found together
We work on the whole stack, so the problem can be wherever it actually is.
Almost always because persist_directory was never set, or it points at an ephemeral container path. The in-memory client is the default, and a path under /tmp is wiped when the container restarts. Point persistence at a mounted volume, move the parquet and SQLite files together whenever the storage is restored, and test that a fresh container can read the existing collection before you trust it.
Chroma runs production RAG for a lot of teams, especially at the small and medium end. The honest limits are concurrent write throughput, horizontal scaling and operational tooling, where Weaviate, Qdrant or Pinecone have had more time to mature. If a single-node server with SQLite under the hood matches your write pattern, it is a perfectly reasonable choice.
In almost every case the embedding path changed. A library upgrade swapped the default embedding function, the chunker started splitting documents at a different boundary, or a retrieval parameter in LangChain drifted. We rebuild a labeled evaluation set, measure recall before touching the system, and isolate the step that regressed.
You pin the client SDK to match the target server, take a cold snapshot of the persist_directory, run the new server pointed at a copy of that data first, confirm the collections load and the queries return sensible results, and only then swap the production traffic. The tenants and databases model has changed enough between minor versions that this is worth doing even when nothing looks broken.
Yes. We inventory the collections and tenants, check whether persistence is actually durable, align the embedding function across read and write paths, rotate any API keys used by the embedding provider, and bring the deployment under version control so the next engineer does not inherit a mystery.
The estimate is free. Describe the deployment, the collection size and what retrieval is doing wrong, and you get a fixed quote before anything touches production.
Free estimate
Tell us what you're running and what's going wrong. A person reads it and comes back with an estimate.
Your message is with a real person, and we'll get back to you soon.
Rather talk now? Call us: