Define "significant". At $lastco, we routed traces to Cassandra and stored them in an AWS elasticsearch domain. Jaeger was used to visualize traces. We also wrote some elasticsearch queries to generate some basic reports, eg finding the most sluggish queries. Pretty standard stuff if you follow the OTEL/jaeger tutorials.
Traces came on the order of hundreds/second, but we didn't have downsampling turned on, just collected all. Traces were saved 7 days (configurable). Very little actual optimization at the point where I left.
I think it cost on the order of dozens to hundreds of dollars a month.
There's an environment variable which you can set on your containers which defines how the tracing sampler behaves. It's in the docs. See OTEL_TRACES_SAMPLER
I think the issue described is when you're storing in a vendor backend, e.g. Data Dog or New Relic. If you're not careful with how much tracing data you ingest you can be looking at significant being 7 figures worth of bills.
Traces came on the order of hundreds/second, but we didn't have downsampling turned on, just collected all. Traces were saved 7 days (configurable). Very little actual optimization at the point where I left.
I think it cost on the order of dozens to hundreds of dollars a month.
There's an environment variable which you can set on your containers which defines how the tracing sampler behaves. It's in the docs. See OTEL_TRACES_SAMPLER
https://opentelemetry.io/docs/specs/otel/configuration/sdk-e...