Project hit its monthly trace limit
A burst of agent traffic or a chatty tool loop quietly burned through the plan allowance, so traces from the rest of the month land nowhere and alerts go quiet.
Quota tuned, sampling by trace type
Running LangSmith?
LangSmith is everywhere in production LLM code built on top of LangChain, and the failure modes are specific: run trees that never close, projects that quietly hit a monthly quota, feedback scores that will not attach to the right trace, and silent sampling that loses the one call you wanted to look at. We fix the plumbing and tune the client so the traces you need are actually there.
What it is
LangSmith is LangChain's hosted observability and evaluation platform for LLM applications. It captures nested run trees from LangChain, LangGraph and plain LLM calls, stores datasets of inputs and outputs, and runs evaluators (deterministic, LLM-as-judge or human) against them.
It is the default observability layer most LangChain users reach for because the integration is invisible when things work. Traces from LangGraph agents, tool calls, retrievers and streaming chat models all land in the same project with the same shape. Pricing is per-trace with a generous free tier, and there is a self-hosted enterprise option for teams that cannot send prompts off their infrastructure.
The honest version
No technology is all upside. This is where LangSmith earns its place and where it tends to cause trouble.
Sound familiar?
The LangSmith problems that bring people to us, and what you have once each one is fixed.
A burst of agent traffic or a chatty tool loop quietly burned through the plan allowance, so traces from the rest of the month land nowhere and alerts go quiet.
Quota tuned, sampling by trace type
Asynchronous tool calls that outlive the parent run, or a context manager that closes too early. The orphaned runs show up in the wrong place or not at all, and token totals look low.
Context carried across async boundaries
Background jobs or worker threads inherit no tracing context, so nested calls either start a new root run or fail to post entirely. Usually a missing contextvar propagation step.
Threads carry the trace context
Scores are written seconds after the run completes but before the trace has finalised server-side, and the API rejects the write. Retries with backoff usually fix it; blind writes do not.
Feedback attaches reliably
An environment that was never updated after a project rename, or a worker running an older env file. Everything looks fine until someone opens the production project and sees nothing recent.
Routing fixed across all services
An evaluator that expected strict JSON while the model returned a code fence, or a schema that drifted after an LLM upgrade. The failures count against scores rather than being surfaced as evaluator bugs.
Evaluators robust to real outputs
A tool whose parameters changed names or types since the dataset was built, so the replay rejects the input before the model is even called.
Dataset examples migrated
A new argument in the OpenAI Python SDK that the LangSmith wrapper has not yet accepted, so plain chat completion calls raise as soon as you add the wrapper.
Client versions aligned
What we do
Bring us one leaking trace, or let us set up the whole LangSmith footprint.
How it works
Tell us what's going wrong. We look at it, then tell you plainly what's wrong and what it will cost.
We fix the cause, not the symptom. If something is down, getting it back up comes first.
Backups, monitoring, updates and access, so the same problem can't quietly come back.
You get a plain-language write-up. Or we stay on for hosting, upkeep and new work.
Often found together
We work on the whole stack, so the problem can be wherever it actually is.
The most common patterns are context lost across async boundaries, background threads that never inherit the parent run, and aggressive sampling that drops exactly the trace you wanted. Check the project matches the one the env var points to, then look for orphaned runs in the project list, then confirm the client library is current.
Sample by trace type instead of blanket sampling, so chatty low-value loops are thinned while user-facing flows stay fully captured. Tag traces with feature names so you can see which product surface is burning quota. Move heavy evaluation runs onto a separate project with its own budget.
Yes. The wrap_openai and wrap_anthropic helpers give you nested run trees against the plain SDKs, and the @traceable decorator works around any Python function. You lose some automatic instrumentation but the data model and UI are identical.
Only if there is a real reason prompt and completion data cannot leave your infrastructure. The self-hosted stack is enterprise-tier, heavy to operate, and priced accordingly. Most teams are better served by the cloud offering with sensible tagging and access controls.
Promote production traces into a labelled dataset, define evaluators that check the output shape and quality, and wire them into CI so a pull request that moves a prompt or model is scored before it merges. The key is keeping evaluators stable even when outputs change in benign ways.
We quote for free. Tell us the symptom, whether it is a missing trace, a quota alert or an evaluator that will not stop failing, and a price comes back before any billing starts.
Free estimate
Tell us what you're running and what's going wrong. A person reads it and comes back with an estimate.
Your message is with a real person, and we'll get back to you soon.
Rather talk now? Call us: