Prometheus has been for a long time the go-to way for monitoring Kubernetes and cloud-native systems. Prometheus provides a full stack, including the metric scraping as well as the backend, including a time-series database for storing the metrics, a UI, AlertManager and more.
Then came OpenTelemetry, which offers a unified way to collect observability telemetry, across traces, logs and metrics. OpenTelemetry is the most active project in the Cloud Native Computing Foundation (CNCF) after Kubernetes, and is rapidly becoming the standard way to collect observability data.
But how do these two hugely popular projects play together?
If people start collecting metrics using OpenTelemetry, can they carry on using Prometheus as their metrics backend?
This is not a tooling issue, but first and foremost a standards misalignment. While Prometheus uses a certain metrics naming convention, OpenTelemetry protocol (OTLP) implements different semantic conventions for metrics, and the two open standards do not fully conform.
This has been the topic of extensive interoperability work done between the two projects under the CNCF.
The next Prometheus release will bring a solution: a recent enhancement enables Prometheus to ingest OpenTelmetry metrics using a new OTLP-compatible ingestion endpoint. The pull request of this feature has just been merged, and will be available as part of the next release of Prometheus. Note that this feature is still experimental, and you will need to turn on the feature flag
otlp-write-receiver to enable it. With this feature on, the OpenTelemetry metrics can be sent on
/otlp/v1/metrics endpoint and ingested natively.
According to Julien Pivotto, core maintainer of Prometheus: “The initial pull request has been merged, paving the way for combined pull-based model and OTLP metrics, so no matter how you generate your metrics, we get you covered.”