hyperdxio/hyperdx
HyperDX is an open source observability platform powered by Clickhouse and OpenTelemetry that centralizes and correlates logs, metrics, traces, errors, and session replays to help engineers quickly resolve production issues.
Awesome ClickHouse › Metrics and Monitoring
flant/loghouse
Loghouse is a ready-to-use log management solution designed specifically for Kubernetes environments, leveraging ClickHouse as its storage backend and providing a web-based user interface for log monitoring and querying. Originally developed in 2017 by Flant, Loghouse addresses the need for efficient log collection, storage, and analysis within Kubernetes clusters. Although active development has ceased as of December 2020, the project remains open source and available for community contributions. The system architecture involves deploying Fluentd as a DaemonSet on each Kubernetes node to collect logs from all pods. Fluentd processes logs at high throughput, capable of handling up to 10,000 log entries per second with moderate resource consumption. Logs are enriched with Kubernetes metadata such as pod names, namespaces, labels, and host information using Kubernetes metadata filters. The processed logs are then inserted into ClickHouse, a columnar database optimized for fast storage and querying of large volumes of data. ClickHouse's efficient storage format allows for minimal disk usage even with billions of log entries. Loghouse supports a simple yet powerful query language that enables users to filter logs by exact key values, regular expressions, and multiple conditions using AND/OR logic. The web UI offers a user-friendly experience similar to Papertrail, with features like customizable time frames, infinite scrolling, query saving, basic user permissions based on Kubernetes namespaces, and CSV export of query results. Deployment is streamlined via Helm charts, requiring a Kubernetes cluster version 1.9 or higher and cert-manager for certificate management. The architecture supports flexible storage options for ClickHouse data, including hostPath volumes and persistent volume claims. Monitoring of Fluentd and ClickHouse is facilitated through Prometheus exporters and Grafana dashboards. While Loghouse is in alpha status, it has been used in production by Flant since 2017. The project roadmap included plans for enhanced filtering, additional deployment options, frontend and backend technology migrations, and CLI tools, though these may not have been realized due to the cessation of active development. Overall, Loghouse provides a comprehensive, scalable, and efficient logging solution tailored for Kubernetes clusters with ClickHouse as the core data store.
https://github.com/flant/loghouse
Also in Metrics and Monitoring
uptrace/uptrace
Uptrace is an open source APM platform that uses OpenTelemetry for data collection and ClickHouse for storage, providing unified monitoring of traces, metrics, and logs with powerful querying, dashboards, and alerting capabilities.
openlit/openlit
OpenLIT is an open-source AI engineering platform providing OpenTelemetry-native observability, monitoring, and management tools for large language models, GPUs, and vector databases, with data stored in ClickHouse for efficient telemetry analysis.
openmeterio/openmeter
OpenMeter is a flexible metering and billing platform for AI, API, and DevOps companies that collects and aggregates usage events in real-time to enable usage-based billing and real-time insights.
trickstercache/trickster
Trickster is an open-source HTTP reverse proxy cache and time series dashboard accelerator that improves performance and scalability for time series databases including ClickHouse, Prometheus, and InfluxDB.
metrico/gigapipe
Gigapipe is a lightweight, polyglot observability stack that supports logs, metrics, traces, and profiling with compatibility for Loki, Prometheus, Tempo, Pyroscope, and OpenTelemetry, powered by ClickHouse and Bun for efficient data ingestion and querying.
Swetrix/swetrix
Swetrix is an open source, privacy-focused, cookie-less web analytics platform that uses Clickhouse for efficient statistical data management and offers self-hosting or cloud deployment options.
burningalchemist/sql_exporter
SQL Exporter for Prometheus is a configuration-driven tool that collects and exposes metrics from various databases, including Clickhouse, for monitoring with Prometheus.