Open Source for Better Observability

Observability as a data analytics problem

Three pillars of Observability

Role of Open Source: Success and Challenges

Open source is the new norm

Tool sprawl is a serious challenge

Relicensing is changing OSS landscape

The leading open source tools for logs, metrics and traces

Open Source Software For Metrics,

  • Prometheus, a CNCF graduate project (second after Kubernetes), is a monitoring system with a dimensional data model, flexible PromQL query language, efficient time series database, and modern alerting approach with AlertManager;
  • OpenMetrics, another CNCF project, offers a format for exposing metrics, which has become a de-facto standard across the industry; and
  • Grafana, a project by Grafana Labs, offering a powerful analytics and visualization tool that’s exceptionally popular in combination with Prometheus.
    Relicensing update: On Apr 2021 Grafana project was relicensed from Apache2 to AGPLv3 by Grafana Labs.

Open Source Software For Logs,

  • ELK Stack, led by Elastic B.V., has been the leading open source choice for a good few years. It is comprised of Elasticsearch text distributed data store, Logstash data collection and processing engine and Kibana visualization tool;
    Relicensing update: On Feb 2021 Elasticsearch and Kibana projects were relicensed from Apache2 to a non-OSS dual license (SSPL and Elastic License) by Elastic B.V.
  • OpenSearch, is a fork of ElasticSearch and Kibana OSS projects, aimed to keep these popular projects open source. The project is led by AWS, which also contributed OpenDistro for Elasticsearch — a set of open source plugins for Elasticsearch; and
  • Loki, led by Grafana Labs, is a log aggregation system specialized for interoperability with Prometheus. Loki doesn’t perform full-text indexing, but rather only indexes labels used in Prometheus.
    Relicensing update: On Apr 2021 Loki project was relicensed from Apache2 to AGPLv3 by Grafana Labs.

Open Source Software For Traces,

  • Jaeger offers a distributed tracing system released as open source by Uber Technologies, which is now a CNCF graduated project;
  • Zipkin, a more veteran Java-based distributed tracing system to collect and look up data from distributed systems; and
  • Skywalking, an open source APM system, including monitoring, tracing, diagnosing capabilities for distributed system in Cloud Native architecture.

Unified telemetry collection with OpenTelemetry



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Dotan Horovits

Dotan Horovits

Technology evangelist, innovation enthusiast, Startup Nation resident, proud father.