Running Weaviate?

Weaviate schema or hybrid search acting up? We sort it out.

A cluster that returned perfect answers in staging has started scoring the wrong passages at the top of the list, the GraphQL endpoint has begun timing out on tenants with the largest shard, and nobody is confident the text2vec module on the schema matches the embedding model the ingest pipeline is actually using. We walk the collection definitions, replay the hybrid queries with alpha fixed, and bring recall back to something you can defend.

  • Since 2015Fixing technology
  • FreeEstimates
  • US teamReal engineers

weaviate.log

still broken

What it is

Weaviate at a glance

Weaviate is an open-source vector database written in Go that stores objects together with their embeddings, so one engine answers both a semantic nearest-neighbor query and a classic structured filter in the same call. It uses the HNSW graph for approximate search by default, exposes a GraphQL API alongside a REST interface, and supports hybrid search where BM25 keyword scoring is fused with vector similarity under a tunable alpha weight.

The project started out of SeMI Technologies in Amsterdam, which later renamed to Weaviate B.V. Vectors can come from an external pipeline or from one of the built-in modules, which call OpenAI, Cohere, Hugging Face, or a local transformer to generate embeddings on write. Production trouble clusters around a handful of patterns: a schema that drifted from the ingest code, HNSW parameters that were never tuned past the quickstart, multi-tenant setups where one tenant has dwarfed the others on a single shard, and hybrid queries whose alpha was picked once and never revisited against real relevance data.

Type
Open-source vector database with hybrid search
First released
2019
License
BSD 3-Clause (open-source core), paid managed cloud
Maintained by
Weaviate B.V. and a community of contributors
Typically used for
RAG retrieval, semantic search, hybrid keyword and vector queries, multi-tenant document search, recommendations

The honest version

Weaviate pros and cons

No technology is all upside. This is where Weaviate earns its place and where it tends to cause trouble.

Where Weaviate is strong

  • Hybrid search without a bolt-onBM25 and vector scores are fused server-side under one alpha parameter, so keyword-sensitive queries stop losing to pure semantic retrieval the moment users search for exact product codes or legal citations.
  • Built-in vectorizer modulestext2vec-openai, text2vec-cohere, text2vec-huggingface and the local transformer module turn ingest into one call, which removes an entire category of pipeline glue code.
  • GraphQL as the query surfaceNested selections, generative-search plugins and aggregation all live in one typed schema, which is nicer to reason about than piecing together REST calls for every variant of a RAG query.
  • Multi-tenancy is a first-class featureEach tenant lives in its own shard with independent indexes, so one noisy customer does not degrade recall or latency for every other account on the cluster.
  • Open-source under BSDSelf-hosting on your own Kubernetes or VMs is a real option, and the managed cloud is a convenience rather than a lock-in gate.
  • Active generative integrationsThe generative-openai, generative-cohere and generative-anthropic modules let you go from retrieval to grounded completion inside a single query when the shape of the app allows it.

Where Weaviate bites

  • HNSW parameters pretend to be defaultsef, efConstruction and maxConnections ship with values that work on toy datasets and quietly cost you recall on millions of vectors. You will need to tune them against your own eval set.
  • GraphQL query planner has sharp edgesDeeply nested selections and hybrid searches on large tenants can time out at the 60-second mark, and the error message rarely points at the specific join that caused it.
  • Autoschema causes long-term painLetting Weaviate infer property types from the first object seen is convenient on day one and a migration headache on day three hundred, especially across environments.
  • Module behavior differs between managed and self-hostedVersioning of the vectorizer modules, the inference endpoints and the generative plugins can drift, so a query that works in Weaviate Cloud sometimes fails against a self-hosted cluster on the same client SDK.
  • Raft and sharding still mature in productionReplication factor, shard count and raft leader behavior have improved with each release but you should expect to read release notes carefully before running mission-critical write paths.
  • Observability is thinner than the engine deservesPrometheus metrics exist, but understanding why a specific tenant is slow usually means reading debug logs and running EXPLAIN-style GraphQL queries by hand.

Sound familiar?

Common Weaviate problems we fix

The Weaviate problems that bring people to us, and what you have once each one is fixed.

ValidationError: vectorizer config missing

The schema and the embedding pipeline disagree

The collection was created with text2vec-openai but the ingest process is uploading pre-computed vectors from a different model, or the other way around. Weaviate complains at write time, or worse, it does not and your results silently drift.

One source of truth for vectors

hybrid search: alpha out of range

Hybrid queries are returning the wrong mix

Alpha set to 1.0 for everything, so BM25 never contributes, or alpha pinned at 0.5 without anyone measuring whether that is right for your corpus. Product codes, SKUs and names score poorly as a result.

Alpha tuned against real relevance labels

GraphQL error: context deadline exceeded

The v1/graphql endpoint keeps timing out

A tenant with millions of objects, a nested where filter on an unindexed property, and an HNSW ef that forces a long walk. The query planner is honest, it just takes too long.

Query shape and indexes aligned

tenant 'acme-prod' holds 4.2M objects on a single shard

One tenant has outgrown multi-tenancy

A large customer landed on the same shard as hundreds of small ones. HNSW rebuilds take hours and small tenants on the same node see latency spikes during write amplification.

Shard plan matched to tenant mix

module 'text2vec-openai' failed: context canceled

Vectorizer modules fail under load

Ingest batches that each call OpenAI once per object exhaust the connection pool or trip the inference rate limit, so writes partially succeed and the collection is left half-vectorized.

Batching that respects the inference tier

shard replication factor 1 on multi-node cluster

No replicas means no safety net

A multi-node cluster set up with RF=1 behaves like three independent databases. A single node failure takes the shard offline, and raft cannot elect around missing data.

Replication factor matched to node count

backup failed: s3 bucket policy denies PutObject

Backups are not actually running

The backup module is configured but the IAM role on the cluster cannot write to the destination bucket, and nobody notices until the first restore test.

Backups tested end to end

autoschema created 'embedding_version'

Autoschema has written properties nobody planned

A rogue ingest job sent a new field, autoschema created it with the wrong data type, and now every subsequent write is being coerced in a way the application does not expect.

Schema explicit, autoschema off

What we do

Weaviate help, from schema design to steady recall

Reach out for one bad query, or let us own the whole retrieval layer.

Schema and tenancy design
Collection boundaries, property types, vectorizer choice and multi-tenancy pattern drawn against the real access shape of your application rather than the quickstart defaults.
Hybrid search tuning
Alpha, fusion type and BM25 weights measured against a labeled relevance set so keyword-heavy queries and semantic queries both land in the right order.
HNSW and sharding
ef, efConstruction, maxConnections and shard count reviewed against measured recall and p99 latency on your own data, with before and after numbers.
Ingest and vectorizer pipelines
Batching, retries and backoff against OpenAI, Cohere or a local transformer so writes finish cleanly without half-indexed objects.
Migrations and upgrades
Moving between Weaviate versions, from self-hosted into Weaviate Cloud or the reverse, with parallel indexes so cutover is a flip rather than a prayer.
Managed Tech
Prometheus dashboards for HNSW health, tenant hot-spotting and backup verification, with engineers who answer the pager when raft trips at 4am.

How it works

From first message to fixed

  1. Look Free estimate

    Tell us what's going wrong. We look at it, then tell you plainly what's wrong and what it will cost.

  2. Fix

    We fix the cause, not the symptom. If something is down, getting it back up comes first.

  3. Harden

    Backups, monitoring, updates and access, so the same problem can't quietly come back.

  4. Hand back or stay on

    You get a plain-language write-up. Or we stay on for hosting, upkeep and new work.

Often found together

Running Weaviate alongside something else?

We work on the whole stack, so the problem can be wherever it actually is.

Questions

Weaviate questions

Something else? Call+1 737 727 7327

Why is our Weaviate recall dropping month over month?

Usually the cause lives upstream of the database. A chunker that changed its splitting rule, an embedding model that silently moved to a new version through an SDK upgrade, or a vectorizer module that is now calling a different inference endpoint will all drop recall without any Weaviate error. We build a small labeled set, measure recall before touching anything, and then isolate which pipeline stage introduced the drift.

Should we use the built-in vectorizer modules or compute embeddings ourselves?

Built-in modules are great for a single-model project because they remove the pipeline, and painful when you want to version your embeddings, batch across many workers, or use a model Weaviate does not bundle. Larger teams tend to compute vectors externally and upload them, which gives full control over cost and versioning at the price of writing the ingest code.

How do we fix a GraphQL query that keeps timing out?

Start by checking which tenant and which property the where filter is hitting. Filters on unindexed properties force a scan of every object, nested selections over generative modules amplify latency, and a low HNSW ef means recall is being bought with wall-clock time. We rewrite the query, add the indexes the plan actually needs, and turn the timeout back off once the real cause is gone.

Can you take over a Weaviate cluster the last team left behind?

Yes, and we do this often. We inventory the collections and tenants, check the schema for autoschema damage, rotate API keys and vectorizer credentials into your account, verify backups actually restore, and bring the configuration under version control before any new changes land.

How do we migrate from Pinecone or Qdrant to Weaviate without downtime?

You create the target schema on Weaviate, write a batched export-and-import that pulls from the source index and upserts into Weaviate with the same IDs, run both stores in parallel behind a feature flag or query shadow, compare recall against a held-out set, and switch the application over when the numbers match. The old index stays live until you have sat on the new one for a week or two.

How much does Weaviate help cost?

Quotes are free. Send us the schema, the query that is misbehaving and a rough object count, and we come back with a fixed price before any work starts.

Free estimate

Weaviate giving you trouble?

Tell us what you're running and what's going wrong. A person reads it and comes back with an estimate.

  • No charge to find out what it will take
  • If it's urgent, say so

Replies come from a person. By sending, you authorize us to email you back. No marketing.