Custom metrics ballooned
A tag like user_id or request_path was added to a metric, each unique value became its own custom metric, and the invoice now has a five-figure line for cardinality nobody intended to buy.
Cardinality back under control
Running Datadog?
Datadog is a brilliant platform and an expensive one, and most of the Datadog work we are called in on is about untangling the second half of that. Custom metrics multiplying, logs ingested at a rate that nobody reviews, alerts configured once and never tuned. We cut the noise, trim the bill, and leave you with dashboards somebody will actually read.
What it is
Datadog is a SaaS monitoring and security platform founded in 2010 by Olivier Pomel and Alexis Lê-Quôc, trading on the Nasdaq since 2019 under the ticker DDOG. It ingests metrics, traces, logs, synthetic tests, real-user monitoring, profiling, security findings and cloud cost data through a single agent and a long list of integrations.
The platform's scope is wide: infrastructure monitoring, APM, log management, network performance, serverless, database monitoring, continuous profiling, cloud security posture management, workload protection, Sensitive Data Scanner, and more. Pricing is per host, per million custom metrics, per gigabyte of logs, per span and per feature, which makes the invoice line-itemed to a degree few observability products match.
The honest version
No technology is all upside. This is where Datadog earns its place and where it tends to cause trouble.
Sound familiar?
The Datadog problems that bring people to us, and what you have once each one is fixed.
A tag like user_id or request_path was added to a metric, each unique value became its own custom metric, and the invoice now has a five-figure line for cardinality nobody intended to buy.
Cardinality back under control
A new product was onboarded with every feature enabled, APM sample rate set to 100% on a chatty service, and log pipelines indexing debug-level logs that nobody reads.
Spend traced to resources, cut cleanly
A service writing JSON logs with large payloads, exceptions duplicated across microservices, or a new log pipeline that forwards everything without filtering. Indexed storage runs out.
Right logs retained, cost predictable
A monitor set up years ago, muted after one false positive, and never unmuted. Meanwhile new monitors fire on anomalies that are just weekend traffic.
Monitors owned, tuned, alive
A service with an aggressive trace rate because somebody wanted the demo to look good, running at full sampling on millions of requests with no dynamic sampling in place.
Sample rate honest for cost and signal
A Datadog cluster agent deployed with incomplete RBAC, so pod metrics, events and tags never make it into Datadog. Dashboards look empty and nobody is sure why.
Cluster agent healthy and authorised
A screen built over two years with every widget somebody wanted added, loading slowly and showing nothing legible on a laptop screen. Usable dashboards are short and deliberate.
Dashboards you can read at a glance
Datadog's CCM module is paid for but was never connected to the AWS billing export, so the engineering team pays for a feature that could cut their AWS bill.
CCM live and informing changes
What we do
Ring us for one noisy monitor, or hand us the Datadog estate and let us trim it properly.
How it works
Tell us what's going wrong. We look at it, then tell you plainly what's wrong and what it will cost.
We fix the cause, not the symptom. If something is down, getting it back up comes first.
Backups, monitoring, updates and access, so the same problem can't quietly come back.
You get a plain-language write-up. Or we stay on for hosting, upkeep and new work.
Often found together
We work on the whole stack, so the problem can be wherever it actually is.
The common culprits are custom metrics ballooning because a high-cardinality tag was added, log indexing enabled on verbose debug logs, APM sample rate left at 100 percent on a chatty service, and features turned on during onboarding that nobody is actively using. We open the usage dashboard with you and trace each major line item to its source.
Find the tags whose unique value counts are the problem, usually user_id, request_path or trace_id. Either aggregate the metric before it is sent, drop the tag at the agent or in the client library, or move the detail into a log where it belongs. The tools in Datadog's Metrics Summary make the offenders easy to spot once you know what to look for.
Both, in most cases, with each one doing what it is good at. Sentry for application errors, source maps and session replay. Datadog for infrastructure, APM, logs and the business-wide dashboards. Overlap is small, and fighting to make one replace the other usually ends up costing more than running both.
Default thresholds that were never tuned, monitors muted long ago and never unmuted, and lots of anomaly monitors on signals that are noisy by nature. We walk the monitor list with the on-call engineers and either tune, retire or route each one, which usually cuts pages by more than half.
Yes. We review each pipeline, move low-value logs to Logging without Limits so they are retained but not indexed, archive long-tail data to S3, and tune the parsers so high-volume log sources are deduplicated or dropped. Rehydration is available when the team needs to search historic data on a specific day.
The first review is free. Share the Datadog organisation and a recent invoice, and we come back with a written plan and a price before we touch any dashboard or monitor.
Free estimate
Tell us what you're running and what's going wrong. A person reads it and comes back with an estimate.
Your message is with a real person, and we'll get back to you soon.
Rather talk now? Call us: