Running Langfuse?

Langfuse traces missing or self-host broken? We unstick it.

Langfuse is excellent until the docker-compose stack falls over, the ingestion worker backs up for an hour, or the ClickHouse volume runs out of space mid-trace. We get the pipeline flowing again, sort the Postgres and ClickHouse layers, and make sure the traces your team relies on actually show up where they should.

  • Since 2015Fixing technology
  • FreeEstimates
  • US teamReal engineers

langfuse-worker.log

still broken

What it is

Langfuse at a glance

Langfuse is an open-source observability platform for LLM applications: it captures traces, generations, scores and evaluations across prompts, chains and agents. Teams use it to debug production prompts, track cost per user, run offline evaluations against datasets and watch latency on model calls.

The project was acquired by ClickHouse in January 2026, and the storage layer leans heavily on ClickHouse MergeTree tables for traces alongside Postgres for metadata and Redis for the ingestion queue. There is a generous self-hosted tier under the MIT license and a managed cloud offering. The architecture is straightforward on paper and awkward on a bad day.

Type
LLM application observability and evaluation
First released
2023
License
MIT core, some commercial features
Maintained by
Langfuse (acquired by ClickHouse, Jan 2026)
Typically used for
Trace capture, prompt debugging, cost tracking, offline evals on LLM apps

The honest version

Langfuse pros and cons

No technology is all upside. This is where Langfuse earns its place and where it tends to cause trouble.

Where Langfuse is strong

  • Open source and self-hostableThe core platform is MIT licensed and runs on your own infrastructure, which matters for teams that cannot send prompts and completions to a third party.
  • Framework-agnostic SDKsPython and TypeScript libraries plug into LangChain, LlamaIndex, Vercel AI SDK, OpenAI and Anthropic without forcing a rewrite of the agent code.
  • Trace model aligns with real agentsNested observations, generations and scores map cleanly onto the way modern agent code actually executes, not just onto single chat completions.
  • Datasets and offline evals built inYou can replay production traces against new prompts and models in the same tool that captured them, which removes a whole class of ad-hoc notebooks.
  • ClickHouse backend scalesThe storage model handles millions of observations without the usual Postgres tears, especially after the v3 architecture moved hot paths off relational tables.
  • Prompt management with versioningPrompts live in Langfuse with labels and versions, so rolling a change forward or back in production does not require a code deploy.

Where Langfuse bites

  • Self-host is more moving parts than the quickstart suggestsPostgres, Redis, ClickHouse, a web container, a worker container and object storage all need to talk to each other and all need capacity planning.
  • Ingestion queue failures are quietIf the worker falls behind or the ClickHouse endpoint hiccups, traces pile up in Redis and the UI looks normal until you run out of memory.
  • Upgrade migrations can be slowClickHouse schema changes on large trace tables take real wall-clock time, and the recommended path is maintenance windows rather than live upgrades.
  • Cost tracking depends on correct model configToken cost calculations rely on a model price registry that has to be kept current for custom or fine-tuned models, or the numbers quietly drift.
  • Access control is coarser than enterprise APMsProject-level roles are fine for small teams; they get awkward when you need per-dataset or per-trace permissioning for regulated workloads.
  • Eval runs compete with ingestion for worker timeOn a single-worker deployment, kicking off a large evaluation can starve the trace ingestion path, so the two workloads usually need to be separated.

Sound familiar?

Common Langfuse problems we fix

The Langfuse problems that bring people to us, and what you have once each one is fixed.

ingestion queue depth 47832, growing

Traces are backed up in Redis

The worker cannot keep pace with incoming observations, usually because ClickHouse is slow, the batch size is wrong, or the worker container is under-provisioned. Memory climbs, dashboards go stale, and eventually traces are lost.

Queue drained, worker right-sized

ClickHouse connection refused

The storage layer is unreachable

A networking rule, a restarted container, or a ClickHouse instance that has crashed and did not come back. The worker retries forever and the UI shows empty projects even though data is being sent.

ClickHouse back and ingesting

parent_observation_id does not exist

Nested spans arrive out of order

A child observation reaches the server before its parent trace has been created, often because the application flushes asynchronously and the parent has not finalised yet. The orphaned spans never surface in the UI.

Spans landing in the correct tree

413 payload too large

Big generations get rejected

A single observation payload exceeds the default limit, usually long tool responses or embedded images in multimodal prompts. The SDK silently drops the batch unless you catch the exception.

Large payloads truncated or chunked

NEXTAUTH_SECRET missing

Self-host will not boot after upgrade

An environment variable that was optional in an older release becomes required, or a schema migration needs a flag the compose file does not pass. The web container crash-loops.

Stack booting on the target version

evaluator timed out after 60s

Evaluations are timing out

An eval that calls a slow model, a dataset that is too large for a single run, or a worker with no GPU budget for model-graded scoring. Results are incomplete and nobody trusts the scores.

Evals finishing with real numbers

MergeTree too many parts

ClickHouse is complaining about compaction

Too many small inserts, retention sweeps not running, or background merges disabled. Query latency rises and the retention job starts to fail silently.

MergeTree healthy, retention running

401 invalid public key

SDKs cannot authenticate

A rotated project key that never made it into the application environment, or a mismatch between public and secret keys between staging and production.

Keys rotated and wired through

What we do

Langfuse help, from rescue to upkeep

Pull us in for a stalled ingestion worker, or let us run the whole observability stack.

Self-host rescue
Docker-compose stacks that will not start, Redis queues that will not drain, or ClickHouse volumes that are out of space. We stabilise the pipeline before anything else.
Trace instrumentation
SDKs wired into LangChain, LlamaIndex, Vercel AI SDK and plain OpenAI or Anthropic calls, with sensible sampling and trace boundaries.
Evaluation pipelines
Datasets, scoring functions and model-graded evals that run in CI rather than being forgotten in a notebook.
Cost and token tracking
Price registries kept current, per-user and per-feature attribution, and dashboards that show where the spend is going before the invoice does.
Upgrades and migrations
Schema migrations rehearsed on a copy of your Postgres and ClickHouse before they run against production data.
Managed Tech
Day-to-day operation of the Langfuse stack, with alerting on queue depth, retention health and ingestion error rates, so you find out before your users do.

How it works

From first message to fixed

  1. Look Free estimate

    Tell us what's going wrong. We look at it, then tell you plainly what's wrong and what it will cost.

  2. Fix

    We fix the cause, not the symptom. If something is down, getting it back up comes first.

  3. Harden

    Backups, monitoring, updates and access, so the same problem can't quietly come back.

  4. Hand back or stay on

    You get a plain-language write-up. Or we stay on for hosting, upkeep and new work.

Often found together

Running Langfuse alongside something else?

We work on the whole stack, so the problem can be wherever it actually is.

Questions

Langfuse questions

Something else? Call+1 737 727 7327

Why are my traces not showing up in Langfuse?

The three usual culprits are the ingestion worker being unable to reach ClickHouse, the SDK silently dropping payloads that exceed the size limit, and project keys that were rotated but never pushed to the app environment. Check the worker logs first, then Redis queue depth, then confirm the public and secret keys match what the SDK is sending.

Should we self-host Langfuse or use Langfuse Cloud?

Self-host when the prompt and completion data must stay on your infrastructure for regulatory or contractual reasons, and you have operational capacity for Postgres, Redis and ClickHouse. Cloud is the right default for everyone else because the moving parts stop being your problem.

How do we track per-customer cost in Langfuse?

Attach a userId and tags to every trace, make sure the model price registry is up to date for every model you call (including fine-tuned deployments), and use the dashboards to group spend by tag. If you call custom endpoints, you may need to set prices manually so token usage is not reported as zero cost.

What happens to Langfuse now that ClickHouse owns it?

The acquisition in January 2026 keeps the open-source project under the same license and the self-hosted path intact, with tighter engineering integration on the storage layer. Teams running on managed cloud see no change to billing or API shape today, but the longer-term roadmap pulls more deeply on ClickHouse primitives.

How do we run offline evaluations on past traces?

You turn production traces into a dataset in Langfuse, define scoring functions (deterministic, LLM-graded or human review), and run them against candidate prompts or models. The results are versioned in the same project, which is why it tends to replace ad-hoc Jupyter notebooks.

How much does Langfuse help cost?

The estimate is free. Send us the symptom, whether it is a stalled worker, a missing trace, or a self-host that will not come up, and we come back with a price before we touch anything.

Free estimate

Langfuse giving you trouble?

Tell us what you're running and what's going wrong. A person reads it and comes back with an estimate.

  • No charge to find out what it will take
  • If it's urgent, say so

Replies come from a person. By sending, you authorize us to email you back. No marketing.