Observability on a Budget for Early‑Stage Teams

TL;DR

#

• A team of six engineers running eight services on three nodes produces roughly 3 GB of logs a day (90 GB/month) and tens of thousands of metric series. Everything below is priced against that one workload.
• At that volume the hosted bill lands around $161/month on Grafana Cloud Pro or $309/month on Datadog with every log indexed — and Datadog drops to about $80/month if you index 10% of your logs instead of all of them. One configuration screen is worth four times the platform choice.
• Grafana Cloud’s free 50 GB of logs runs out on day 17 of that month. Free tiers are a real answer for a smaller team, not for this one.
• Logs cost roughly ten times the bytes of metrics at this workload, but at list prices the metrics line can still be the bigger one — because metrics are billed per series and logs per byte. Different mistakes, different files.
• The four levers that move the bill, in order of effect: index less, retain less, log less, label less.
• Prices below were checked against vendor pricing pages on 7 August 2026. They move. Re-check before you commit.

Who this guide is for

#

You are 3-10 engineers shipping one product, with no dedicated SRE and no observability budget line that anyone has defended. You want to know when production breaks, be able to debug it, and not discover a four-figure invoice in month three.

This guide is about the money. If your question is “should I run this myself instead”, that is a different calculation and it lives in SaaS vs self-hosted monitoring.

The reference workload

#

Everything in this guide is priced against one made-up-but-arithmetically-honest team. Substitute your own numbers; the method matters more than the totals.

The log figure comes out of the request rate, not out of a survey:

• 40 req/s × 86,400 s = 3.5 million access log lines a day.
• Roughly half a line of application logging per request (job starts, warnings, retries) = 1 million more.
• Ingress controller, kubelet, control plane, CI runners and cron = call it 0.5 million more.
• 5 million lines × 600 bytes = 3 GB/day, 90 GB over a 30-day month.

Measure yours instead of trusting mine

#

Two commands and one query replace the whole table above.

# Bytes in a single log line from a representative service
kubectl logs deploy/your-api --tail=1 | wc -c

# Bytes per minute a pod actually emits
kubectl logs deploy/your-api --since=1m | wc -c

For metrics, ask Prometheus how many series it is holding:

count({__name__=~".+"})

That number is the one that surprises people. A default kube-prometheus-stack install on three nodes brings node-exporter, kube-state-metrics and the operator’s own rules along with your application metrics, and the total lands in the tens of thousands before you have instrumented anything interesting. Measure it before you sign anything priced per series.

What the reference workload costs on hosted platforms

#

All prices checked 7 August 2026 on the vendors’ own pricing pages.

Grafana Cloud Pro

#

Grafana Cloud bills logs across four separate dimensions, which is unusual and worth understanding because three of them will be zero for you.

Note which line is largest. Ninety gigabytes of logs cost $20.50; twenty-five thousand metric series cost $97.50. That is not a typo and it is the single most useful thing on this page.

Datadog

#

Datadog splits logs into ingest (cheap, everything) and indexing (expensive, only what you can search). The default of indexing everything is what produces the invoices people write blog posts about.

Standard indexing offers 3, 7, 15 or 30-day retention. APM is a separate $31/host/month on top when bought with Infrastructure. Container counts above the per-host allowance are billed separately, so check that line for a Kubernetes cluster.

The lesson generalises past Datadog: on any per-event pricing model, the exclusion filter is worth more than the vendor negotiation.

The others, briefly

#

• New Relic includes 100 GB of data ingest per month free, which swallows this team’s entire log volume — but metrics count against the same pool, so measure before you assume. The bill is driven by seats instead: one free full platform user, then $49/user for Core, or $10 for the first full platform user and $99 for each additional one (maximum five) on Standard. Basic users are free and unlimited.
• Better Stack charges $0.15/GB ingested and $0.08/GB/month retained, so 90 GB at 30-day retention is roughly $21/month for logs. On-call responder licences are $34/month each on monthly billing.
• Axiom free covers 500 GB/month of data loading but caps storage at 25 GB, which 90 GB/month at 30-day retention will not fit inside. Axiom Cloud at $25/month includes 1 TB/month of loading and 100 GB of storage.
• Honeycomb free covers 20 million events per month with unlimited seats. The reference workload’s 150 million monthly log events is well past that, but Honeycomb is an events-and-traces product, not a log sink — sample before you compare.

What fits inside a free tier

#

Free tiers are not a trick. For a smaller team they are the correct answer for a year or more. Here is where each one stops, as of 7 August 2026.

The reference team blows through Grafana Cloud’s free 50 GB of logs on day 17 of the month (50 ÷ 3), and its 25,000 series were already 2.5× the free series limit on day one. A team half its size — four services, 1.5 GB/day — fits the log allowance comfortably and still breaks the series limit. For most small teams the free tier’s real ceiling is cardinality, not gigabytes.

Why logs cost more than metrics, and when they don’t

#

Per byte stored, this is not close.

• 25,000 series scraped every 15 seconds is 1,667 samples/second, or 144 million samples/day. Prometheus documents an average of 1-2 bytes per sample on disk, so that is roughly 288 MB/day.
• The same team’s logs are 3 GB/day. About ten times the bytes.

And the log bytes are the harder ten. A metric sample is a float in a compressed block that nobody ever reads individually. A log line has to be searchable, which means something has to index it, hold it hot, and answer a grep for a stack trace at 3 a.m. That is why hosted log products bill write, retention and query as separate line items while metrics are a single number.

But the pricing models pull in opposite directions, and this is where teams get caught:

• Metrics are billed by shape. You pay per active series, so cost tracks your label design, not your traffic. Adding user_id to one histogram can multiply a 200-series metric by your user count. At Grafana Cloud’s $6.50 per 1,000 series, an extra 100,000 series is $650/month from a one-line change that passed code review.
• Logs are billed by volume. You pay per gigabyte or per event, so cost tracks your traffic and log level. Turning on debug logging in production multiplies the bill by however chatty that service is.

So at the reference workload, on Grafana Cloud list prices, the metrics line is five times the log line. At Datadog’s per-event indexing, the log line is five times everything else. Both statements are true simultaneously, which is why “logs are the expensive one” is folklore rather than guidance. Price your own two numbers against your own vendor’s two models.

The four levers, ranked by how much they save

#

1. Index less. On per-event pricing this is the biggest single lever available to you — the Datadog example above is a 4× difference from exclusion filters alone. Health checks, readiness probes, static asset requests and successful 2xx access logs are the usual first cuts: keep them in ingest and archives, keep them out of the search index.
2. Retain less. Default to 7-30 days and raise it only for a named reason. Grafana Cloud Pro includes 30 days and charges $0.10/GB for each additional 30-day increment; on AWS CloudWatch, ingest is $0.50/GB but storage afterwards is $0.03/GB/month, so retention is rarely where your CloudWatch money goes. Know which of the two your vendor charges for before you tune it.
3. Log less. Debug off in production. One line per request, not four. Structured JSON with a fixed field set — service, env, version, level, message, trace_id — so you can filter cheaply instead of scanning. Long stack traces and base64 payloads are worth truncating at the collector.
4. Label less. No user_id, request_id, session_id, email address or full URL path in a metric label. Those belong in logs and traces, where they are cheap. In metrics they are the $650/month mistake above.

Notice the order. Teams reach for lever 3 first because it feels like discipline, and it is the slowest and most annoying of the four.

How the bill spikes, and the guardrails that catch it

#

The realistic failure is not gradual growth. It is a service crash-looping at DEBUG over a long weekend.

Take the reference team and assume one service goes ten times louder for three days: an extra 81 GB of logs.

• Grafana Cloud: 81 GB × ($0.05 process + $0.40 write) = about $36 extra. Annoying, not alarming.
• Datadog with everything indexed: 135 million extra events × $1.70/million = about $230 extra, on top of a $309 baseline.

Per-event pricing punishes storms roughly five times harder than per-gigabyte pricing at this line size. That is a real, checkable difference between the two models and it should inform which one you pick.

Guardrails worth setting on day one:

• A hard quota on the index, not just an alert. Datadog supports a daily quota per log index: once it is reached, “logs are no longer indexed but are still available in the livetail, sent to your archives, and used to generate metrics from logs.” The quota resets at 14:00 UTC by default and the reset time is configurable. That is a spending cap that still leaves you the data.
• Exclusion filters written before you need them, so the storm hits a filter instead of the index.
• A billing alert on the observability vendor itself. The one bill nobody monitors is the monitoring bill.
• A max_line_size-style truncation at the collector, so a single 40 MB stack trace is not a line item.

The signals worth paying for

#

Cost control is only half the job. The other half is not paying for signals you never look at.

Four golden signals per service

#

• Latency: p95/p99 on a few key endpoints
• Errors: 5xx rate, plus a handful of business-critical application errors
• Traffic: requests per second
• Saturation: CPU, memory, disk, and DB connections if they bite you

That is the whole set — four, not five. Availability is not a fifth golden signal; it is what you derive from errors and traffic. Add an external uptime check anyway, because it catches the case where your service is too broken to report any metrics at all. UptimeRobot’s free tier does 50 monitors at 5-minute intervals, so this line costs nothing.

Five to ten alerts

#

If you have more than ten, you are alerting on causes instead of symptoms. A symptom is what a user feels: checkout is erroring, p95 doubled. A cause is a machine detail that may or may not lead there: CPU at 90%, a pod restarted, a queue is deep. There are hundreds of causes and most resolve themselves. Page on the handful of symptoms; use the causes to debug once you are already awake.

Prefer “sustained for N minutes” over instantaneous thresholds, give every alert an owner and a documented action, and route everything to one place. What to do once one fires is Incident response for small teams.

The bar you are aiming for

#

One place to look during an incident that holds both logs and metrics. Five to ten alerts worth waking up for. Deploy markers on the dashboards so you can see what changed. Explicit retention and cost limits. That is the whole bar, and teams usually fail it on the alert count rather than on tooling.

When to add tracing

#

Not yet, probably. Add it when incidents involve call chains you cannot reason about from logs, or when latency regressions stop being explainable from metrics. Then roll it out narrowly: one critical request path, one or two services, sampled — not every service at 100%.

Instrument with OpenTelemetry rather than a vendor SDK. It costs the same to write and it means the next section of this guide (switching vendors when the bill moves) does not require re-instrumenting anything.

A two-to-four week plan

#

1. Measure first. Run the three commands in “Measure yours” above. Write down GB/day and active series. Do not price anything before you have those two numbers.
2. Pick on the model, not the brand. If your logs are high-volume and low-value, prefer per-gigabyte pricing. If they are low-volume and you search all of them, per-event indexing is fine.
3. Set the guardrails before the first log arrives: retention, exclusion filters, a daily index quota, a billing alert.
4. Ship structured logs with the fixed field set, to one sink.
5. Build one golden-signals dashboard plus an external uptime check.
6. Add 5-10 alerts, each with an owner and an action.
7. Run one tabletop incident and confirm you can find a cause using only what you are paying for. Anything you did not open is a candidate for the next round of cuts.

Questions people ask

#

How much does observability cost for a small startup?

#

For a team of six engineers running eight services that produce about 3 GB of logs a day and 25,000 metric series, the hosted bill lands around $161/month on Grafana Cloud Pro or around $309/month on Datadog with every log indexed, based on list prices checked on 7 August 2026. Turning on exclusion filters so that only 10% of logs reach the search index cuts the Datadog figure to roughly $80/month, which is a bigger saving than switching vendors. Below roughly 1.5 GB of logs a day, free tiers from Grafana Cloud, New Relic or Axiom will cover a team’s logs outright. The number that decides your bill is not the vendor, it is gigabytes per day and active metric series.

How do I estimate how many gigabytes of logs my team produces?

#

Multiply your average request rate by the seconds in a day to get access log lines, add roughly half a line of application logging per request, then add platform noise from the ingress controller, kubelet and CI. Multiply that line count by your average line size, which you can measure directly with kubectl logs deploy/your-api --tail=1 | wc -c — structured JSON lines with Kubernetes labels attached typically run several hundred bytes. Forty requests per second at 600 bytes per line works out to about 3 GB a day, or 90 GB a month. Do this before you look at any pricing page, because every vendor’s pricing page is meaningless without that one number.

Why is log storage more expensive than metrics?

#

Per byte, metrics are far smaller: Prometheus averages 1-2 bytes per sample on disk, so 25,000 series scraped every 15 seconds is around 288 MB a day, against 3 GB a day of logs from the same system. Log bytes are also harder bytes, because logs have to stay searchable, which is why hosted vendors bill log write, retention and query as separate line items while metrics are one number. But the pricing models pull in opposite directions: metrics are billed per active series, so their cost tracks your label design, while logs are billed per gigabyte or per event, so their cost tracks your traffic and log level. At Grafana Cloud list prices, 25,000 series can cost five times more than 90 GB of logs, so “logs are the expensive one” is folklore rather than a rule.

What is the cheapest way to get logs, metrics and alerts?

#

Free tiers, honestly, for as long as they last: as of 7 August 2026 Grafana Cloud’s free plan covers 10,000 active series, 50 GB of logs and 14-day retention for three users, New Relic includes 100 GB of monthly ingest with unlimited basic users, and UptimeRobot gives 50 uptime monitors at 5-minute intervals. The limit you hit first is almost always the metric series count, not the log volume, because a default Kubernetes monitoring install produces tens of thousands of series before you instrument anything. When you outgrow free, the cheapest paid path is a per-gigabyte log product with short retention rather than a per-event indexing product with everything indexed. Self-hosting is not the cheap option at this scale — that calculation is in the SaaS versus self-hosted comparison.

How do I stop a monitoring bill from spiking?

#

Set a hard quota, not just an alert: Datadog’s daily index quota stops indexing once reached while still delivering logs to livetail, archives and log-based metrics, and it resets at 14:00 UTC by default. Write exclusion filters for health checks, readiness probes and successful 2xx access logs before you need them, so a crash loop hits a filter instead of your index. Put a billing alert on the observability vendor itself, because the monitoring bill is the one nobody monitors. And truncate oversized lines at the collector, so a single enormous stack trace never becomes a line item.

Do I need distributed tracing yet?

#

Probably not. Tracing earns its cost when incidents involve call chains you cannot reconstruct from logs, or when latency regressions stop being explainable from metrics alone — until then it is a second bill for information your logs already carry. When you do add it, start narrow: one critical request path, one or two services, sampled rather than complete. Instrument with OpenTelemetry rather than a vendor SDK, so that changing backends later is a collector config change instead of a re-instrumentation project.

Related reads

#

• SaaS vs self-hosted monitoring — the same workload priced both ways, and where the crossover actually sits
• Kubernetes monitoring stack — Prometheus, Grafana, Loki and Alertmanager with the Helm values that make them fit a small cluster
• Incident response for small teams — what happens after one of those 5-10 alerts fires
• Minimal DevOps stack — where observability sits among everything else you have to run
• Open source DevOps tools — the shortlist, including why open source and self-hosted are not the same choice