When the tool can't tell you: Adaptive Sampling and a mistyped metric

Most of what a Datadog cost diagnostic does is comparison: this month against last month, this filter's rate against its actual matched volume, this deploy date against that usage step-change. Comparisons need two numbers. Every so often, one of the two numbers isn't available at all — not because the finding doesn't matter, but because Datadog's own API surface doesn't expose a reliable way to get it. A recent beta run against a workforce-scheduling SaaS company's org hit exactly this twice in one report, and both times the tool did the same thing: it stopped trying to determine an answer, and drafted the support-ticket question instead.

Finding one: the Adaptive Sampling Target it couldn't find

Datadog's Adaptive Sampling Target is a real, useful setting — it governs how aggressively the platform self-adjusts trace sampling to hit a target ingestion volume. The tool tried to determine this org's configured target automatically, using the approach that works in most orgs: look for a retention filter whose name follows the known Adaptive Sampling naming convention, and read the target from there. In this org, nothing matched. No filter in the list followed that naming pattern.

That's not the same as "there is no Adaptive Sampling Target configured." It's "we couldn't confirm one from the outside, using the one indirect signal we know how to read." Those are different claims, and the report was careful to only make the second one. Rather than assert a target that might be wrong, or silently drop the check and say nothing, it drafted this:

"Can you confirm whether the APM Adaptive Sampling Target is set correctly? I wasn't able to match any retention filter to the known adaptive-sampling naming convention."

That's a real, sendable support-ticket question — specific enough that a Datadog support engineer can answer it directly, honest enough that it doesn't claim to already know the answer.

Finding two: a metric named "couter"

The second inconclusive finding was a custom metric named couter — almost certainly a typo of "counter," though the tool doesn't assume that, because assuming it is itself a guess. Its required tags couldn't be determined. Rather than report "this metric is missing tags X and Y" (a specific claim it wasn't in a position to make) or silently skip a metric it couldn't fully resolve, it drafted:

"For the custom metric 'couter,' tags could not be determined. Is this a typo, a missing tag, or a config issue that we need to correct?"

That question is deliberately open to all three explanations, because all three are plausible and the tool has no basis to rule any of them out from the outside. A typo in a metric name is a mundane thing, but it's a real, if unglamorous, source of unauditable spend: a metric nobody can confidently identify is a metric nobody can confidently tag, and a metric nobody can confidently tag is a metric whose cost nobody can confidently attribute to a service or environment — the same underlying problem covered in our post on untagged custom metrics, just arriving from a different direction. There, the metric was correctly named and simply untagged. Here, the name itself might be the whole problem.

Why this matters more than either finding individually

Datadog's usage and configuration APIs are genuinely comprehensive for a lot of what a cost diagnostic needs — estimated_cost, hourly_usage, retention filter configs, synthetic test definitions are all queryable, structured, and reliable once you know the specific shape each endpoint expects. But "comprehensive for most things" is not the same as "complete for everything," and Adaptive Sampling naming conventions are a good example of the gap: there's no documented, stable contract for programmatically confirming a target from outside the Agent/tracer configuration itself. A tool that pretends otherwise — that infers a target from indirect signals and reports it as fact — is going to be confidently wrong some fraction of the time, and confidently wrong is worse than openly uncertain, because it looks the same as being right until someone acts on it.

The alternative isn't silence, either. Silently dropping a check because the data isn't available just means the org never finds out there was a question worth asking. A drafted support ticket sits in between: it doesn't overclaim, and it doesn't give up. It converts "we don't know" into "here's exactly what to ask the people who do."