ClickHouse: Breaking the Speed Limit for Observability and Analytics
ClickHouse® is a rising star in observability and analytics, challenging performance conventions with its breakneck speed. This open source OLAP column store, originally developed at Yandex to power their web analytics platform at massive scale, has quickly evolved into one of the hottest open source observability data stores around, primarily because it’s fast — really fast.
On my recent OpenObservability Talks episode I invited Robert Hodges, CEO of Altinity — the second largest contributor to the ClickHouse project—to hear all about the project. Altinity also just released a new open source project that extends ClickHouse with Apache Iceberg shared storage, unlocking dramatic improvements in both performance and cost efficiency. Think 90% reductions in storage costs and 10 to 100x faster queries, all without requiring any changes to your existing applications.
Let’s unpack what makes ClickHouse tick, and why it’s gaining momentum across some of the most data-intensive organizations on the planet.
Under the Hood: Speed by Design
ClickHouse is an SQL-based column-oriented OLAP (Online Analytical Processing) database that excels at real-time analytical queries over massive datasets. Originally developed by Yandex engineering to power Yandex.Metrica, the second largest web analytics platform in the world, where it was used for tracking about 12 billion page views and clicks daily (As of April 2014).
ClickHouse achieves its signature speed through deliberate architectural choices. It’s a columnar store, so queries only scan the columns they need — dramatically reducing I/O. It leverages modern CPUs with SIMD vectorized execution and processes data in parallel, maximizing throughput.
“ClickHouse isn’t a black box — it’s a performance machine with its hood open,” Hodges explained. “What you see is an engine that’s optimized down to the CPU cache line.”
Data is stored in compressed blocks, indexed for fast filtering, and written in an append-only fashion, allowing concurrent reads and writes with minimal locking. MergeTree engines handle replication, partitioning, and automatic background merging — all while queries stay snappy.
ClickHouse Community and Ecosystem
In 2016 Yandex open sourced ClickHouse under Apache 2.0 license. Then In 2021 Yandex spun off ClickHouse into a standalone company, ClickHouse Inc. This company now owns the project and is the main driving force behind it, with the original maintainers as part of the team.
The ClickHouse project has gathered immense popularity, with nearly 41k GitHub stars and over 7k forks to date. It has also gathered a vibrant community around it beyond ClickHouse Inc, with contributors from Altinity and others. “In any given year, ClickHouse gets merged PRs from hundreds of people.” shares Robert, “there are thousands of people who log issues.”
ClickHouse is used for a variety of use cases, where speed matters, from Ad networks and real-time marketing to DNS networks and high frequency trading, by companies such as Deutsche Bank, Bloomberg, Amadeus and Cloudflare.
One prominent use case is a favorite of mine: Observability. “There’s at least a couple dozen major use cases, “ says Robert, “I think the big ones today are observability.”
Observability: Where ClickHouse Shines
Some observability tools like SigNoz and VictoriaLogs already use ClickHouse natively. Others, like Shopify and Uber, have rolled out bespoke observability platforms with ClickHouse at the core, leveraging its speed and compression to process billions of events per second. In fact, I covered Shopify’s observability journey on a separate blog.
Logs, being time-ordered, append-only, highly compressible telemetry data, and similarly, spans—trace data—are well suited, with ClickHouse’s ability to ingest data from Kafka, aggregate on the fly, and support rapid queries over months of observability data. Uber’s schema-agnostic log analytics implementation based on ClickHouse is an interesting example of that.
With OpenTelemetry becoming the default telemetry pipeline, ClickHouse slots in perfectly as a backend for high-volume, low-latency observability platforms.
Kubernetes-Native and Production-Ready
ClickHouse plays well with Kubernetes thanks to the open-source ClickHouse Operator maintained by Altinity. It manages deployment, upgrades, backups, and replication — all the heavy lifting required to run ClickHouse reliably in a production-grade cloud environment.
“Running ClickHouse in production is like flying a jet,” said Hodges. “You need the right instrumentation and controls, especially in a Kubernetes environment.”
For storage, fast local disks or SSD-backed block volumes are key. Network storage will likely become a bottleneck. Combine that with proper pod placement policies and anti-affinity rules to spread replicas well, and you’ll have a resilient ClickHouse cluster that scales predictably.
It’s important to note that in order to get a smooth ride with ClickHouse, without query crashes and out-of-memory errors, and to get the superior performance we talked about, you’d need to know what’s going on inside, perhaps unlike experiences you’d know from the likes of Snowflake. Robert explains it using the race car metaphor: “You’re going to be opening up the hood and you’re kind of digging under the carburetor and, you know, like, you know, sort of unbolting the transmission to fix something. It is not like Snowflake.”
Extending ClickHouse with Apache Iceberg Data Lake
As data volumes grow, the challenge of cost becomes more prominent, and so is the pressure to use cheap storage. Last month Altinity launched Project Antalya, a new initiative that integrates ClickHouse with Apache Iceberg, a popular open table format for data lakes, allowing ClickHouse to operate directly on object storage like S3, while separating compute from storage.
This means ClickHouse can now support stateless compute clusters that spin up and down as needed — dramatically improving elasticity and reducing storage costs. It also opens the door for shared access to data across multiple compute engines, fitting neatly into modern lakehouse architectures.
Want to learn more? Check out the OpenObservability Talks episode: ClickHouse: Breaking the Speed Limit for Observability and Analytics.
